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Preface 



The engineering objective of high performance control using the tools of optimal 
control theory, robust control theory, and adaptive control theory is more achiev- 
able now than ever before, and the need has never been greater. Of course, when 
we use the term high performance control we are thinking of achieving this in 
the real world with all its complexity, uncertainty and variability. Since we do not 
expect to always achieve our desires, a more complete title for this book could be 
“Towards High Performance Control”. 

To illustrate our task, consider as an example a disk drive tracking system for a 
portable computer. The better the controller performance in the presence of eccen- 
tricity uncertainties and external disturbances, such as vibrations when operated 
in a moving vehicle, the more tracks can be used on the disk and the more memory 
it has. Many systems today are control system limited and the quest is for high 
performance in the real world. 

In our other texts Anderson and Moore (1989), Anderson and Moore (1979), 
Elliott, Aggoun and Moore (1994), Helmke and Moore (1994) and Mareels and 
Polderman (1996), the emphasis has been on optimization techniques, optimal es- 
timation and control, and adaptive control as separate tools. Of course, robustness 
issues are addressed in these separate approaches to system design, but the task 
of blending optimal control and adaptive control in such a way that the strengths 
of each is exploited to cover the weakness of the other seems to us the only way 
to achieve high performance control in uncertain and noisy environments. 

The concepts upon which we build were first tested by one of us, John Moore, 
on high order NASA flexible wing aircraft models with flutter mode uncertainties. 
This was at Boeing Commercial Airplane Company in the late 1970s, working 
with Dagfinn Gangsaas. The engineering intuition seemed to work surprisingly 
well and indeed 180° phase margins at high gains was achieved, but there was 
a shortfall in supporting theory. The first global convergence results of the late 
1970s for adaptive control schemes were based on least squares identification. 
These were harnessed to design adaptive loops and were used in conjunction with 
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linear quadratic optimal control with frequency shaping to achieve robustness to 
flutter phase uncertainty. However, the blending of those methodologies in itself 
lacked theoretical support at the time, and it was not clear how to proceed to 
systematic designs with guaranteed stability and performance properties. 

A study leave at Cambridge University working with Keith Glover allowed 
time for contemplation and reading the current literature. An interpretation of the 
Youla-Kucera result on the class of all stabilizing controllers by John Doyle gave 
a clue. Doyle had characterized the class of stabilizing controllers in terms of a 
stable filter appended to a standard linear quadratic Gaussian LQG controller de- 
sign. But this was exactly where our adaptive filters were placed in the designs 
we developed at Boeing. Could we improve our designs and build a complete 
theory now? A graduate student Teng Tiow Tay set to work. Just as the first simu- 
lation studies were highly successful, so the first new theories and new algorithms 
seemed very powerful. Tay had also initiated studies for nonlinear plants, conve- 
niently characterizing the class of all stabilizing controllers for such plants. 

At this time we had to contain ourselves not to start writing a book right 
away. We decided to wait until others could flesh out our approach. Iven Mareels 
and his PhD student Zhi Wang set to work using averaging theory, and Roberto 
Horowitz and his PhD student James McCormick worked applications to disk 
drives. Meanwhile, work on Boeing aircraft models proceeded with more con- 
servative objectives than those of a decade earlier. No aircraft engineer will trust 
an adaptive scheme that can take over where off-line designs are working well. 
Weiyong Yan worked on more aircraft models and developed nested-loop or it- 
erated designs based on a sequence of identification and control exercises. Also 
Andrew Paice and Laurence Irlicht worked on nonlinear factorization theory and 
functional learning versions of the results. Other colleagues Brian Anderson and 
Robert Bitmead and their coworkers Michel Gevers and Robert Kosut and their 
PhD students have been extending and refining such design approaches. Also, 
back in Singapore, Tay has been applying the various techniques to problems 
arising in the context of the disk drive and process control industries. 

Now is the time for this book to come together. Our objective is to present the 
practice and theory of high performance control for real world environments. We 
proceed through the door of our research and applications. Our approach special- 
izes to standard techniques, yet gives confidence to go beyond these. The idea is 
to use prior information as much as possible, and on-line information where this is 
helpful. The aim is to achieve the performance objectives in the presence of vari- 
ations, uncertainties and disturbances. Together the off-line and on-line approach 
allows high performance to be achieved in realistic environments. 

This work is written for graduate students with some undergraduate back- 
ground in linear algebra, probability theory, linear dynamical systems, and prefer- 
ably some background in control theory. However, the book is complete in itself, 
including appropriate appendices in the background areas. It should appeal to 
those wanting to take only one or two graduate level semester courses in control 
and wishing to be exposed to key ideas in optimal and adaptive control. Yet stu- 
dents having done some traditional graduate courses in control theory should find 
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that the work complements and extends their capabilities. Likewise control engi- 
neers in industry may find that this text goes beyond their background knowledge 
and that it will help them to be successful in their real world controller designs. 
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Performance Enhancement 



1 . 1 Introduction 

Science has traditionally been concerned with describing nature using mathemati- 
cal symbols and equations. Applied mathematicians have traditionally been study- 
ing the sort of equations of interest to scientists. More recently, engineers have 
come onto the scene with the aim of manipulating or controlling various pro- 
cesses. They introduce (additional) control variables and adjustable parameters to 
the mathematical models. In this way, they go beyond the traditions of science and 
mathematics, yet use the tools of science and mathematics and, indeed, provide 
challenges for the next generation of mathematicians and scientists. 

Control engineers, working across all areas of engineering, are concerned with 
adding actuators and sensors to engineering systems which they call plants. They 
want to monitor and control these plants with controllers which process informa- 
tion from both desired responses (commands) and sensor signals. The controllers 
send control signals to the actuators which in turn affect the behavior of the plant. 
They are concerned with issues such as actuator and sensor selection and location. 
They must concern themselves with the underlying processes to be controlled and 
work with relevant experts depending on whether the plant is a chemical system, 
a mechanical system, an electrical system, a biological system, or an economic 
system. They work with block diagrams, which depict actuators, sensors, proces- 
sors, and controllers as separate blocks. There are directed arrows interconnecting 
these blocks showing the direction of information flow as in Figure 1.1. The di- 
rected arrows represent signals, the blocks represent functional operations on the 
signals. Matrix operations, integrations, and delays are all represented as blocks. 
The blocks may be (matrix) transfer functions or more general time-varying or 
nonlinear operators. 

Control engineers talk in terms of the controllability of a plant (the effective- 
ness of actuators for controlling the process), and the observability of the plant 
(the effectiveness of sensors for observing the process). Their big concept is that 
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Disturbances 




FIGURE 1.1. Block diagram of feedback control system 



of feedback, and their big challenge is that of feedback controller design. Their 
territory covers the study of dynamical systems and optimization. If the plant is not 
performing to expectations, they want to detect this under-performance from sen- 
sors and suitably process this sensor information in controllers. The controllers in 
turn generate performance enhancing feedback signals to the actuators. How do 
they do this? 

The approach to controller design is to first understand the physics or other sci- 
entific laws which govern the behavior of the plant. This usually leads to a math- 
ematical model of the process, termed a plant model. There are invariably aspects 
of plant behavior which are not captured in precise terms by the plant model. 
Some uncertainties can be viewed as disturbance signals, and/or plant parameter 
variations which in turn are perhaps characterized by probabilistic models. Un- 
modeled dynamics is a name given to dynamics neglected in the plant model. Such 
are sometimes characterized in frequency domain terms. Next, performance mea- 
sures are formulated in terms of the plant model and taking account of uncertain- 
ties. There could well be hard constraints such as limits on the controls or states. 
Control engineers then apply mathematical tools based in optimization theory to 
achieve their design of the control scheme. The design process inevitably requires 
compromises or trade-offs between various conflicting performance objectives. 
For example, achieving high performance for a particular set of conditions may 
mean that the controller is too finely tuned, and so can not yet cope with the con- 
tingencies of everyday situations. A racing car can cope well on the race track, 
but not in city traffic. 

The designer would like to improve performance, and this is done through in- 
creased feedback in the control scheme. However, in the face of disturbances or 
plant variations or uncertainties, increasing feedback in the frequency bands of 
high uncertainty can cause instability. Feedback can give us high performance for 
the plant model, and indeed insensitivity to small plant variations, but poor per- 
formance or even instability of the actual plant. The term controller robustness is 
used to denote the ability of a controller to cope with these real world uncertain- 
ties. Can high performance be achieved in the face of uncertainty and change? 
This is the challenge taken up in this book. 
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1.2 Beyond Classical Control 

Many control tasks in industry have been successfully tackled by very simple 
analog technology using classical control theory. This theory has matched well 
the technology of its day. Classical three-term-controllers are easy to design, are 
robust to plant uncertainties and perform reasonably well. However, for improved 
performance and more advanced applications, a more general control theory is 
required. It has taken a number of decades for digital technology to become the 
norm and for modern control theory, created to match this technology, to find its 
way into advanced applications. The market place is now much more competitive 
so the demands for high performance controllers at low cost is the driving force for 
much of what is happening in control. Even so, the arguments between classical 
control and modern control persist. Why? 

The classical control designer should never be underestimated. Such a person 
is capable of achieving good trade-offs between performance and robustness. Fre- 
quency domain concepts give a real feel for what is happening in a process, and 
give insight as to what happens loop-by-loop as they are closed carefully in se- 
quence. An important question for a modem control person (with a sophisticated 
optimization armory of Riccati equations and numerical programming packages 
and the like) to ask is: How can we use classical insights to make sure our mod- 
ern approach is really going to work in this situation? And then we should ask: 
Where does the adaptive control expert fit into this scene? Has this expert got to 
fight both the classical and modern notions for a niche? 

This book is written with a view to blending insights and methods from classi- 
cal, optimal, and adaptive control so that each contributes at its point of strength 
and compensates for the weakness of others, so as to achieve both robust control 
and high performance control. Let us examine these strengths and weaknesses in 
turn, and then explore some design concepts which are perhaps at the interface of 
all three methods, called iterated design, plug-in controller design, hierarchical 
design and nested controller design. 

Some readers may think of optimal control for linear systems subject to qua- 
dratic performance indices as classical control, since it is now well established 
in industry, but we refer to such control here as optimal control. Likewise, self- 
tuning control is now established in industry, but we refer to this as adaptive con- 
trol. 



Classical Control 

The strength of classical control is that it works in the frequency domain. Distur- 
bances, unmodeled dynamics, control actions, and system responses all predom- 
inate in certain frequency bands. In those frequency bands where there is high 
phase uncertainty in the plant, feedback gains must be low. Frequency character- 
istics at the unity gain cross-over frequency are crucial. Controllers are designed 
to shape the frequency responses so as to achieve stability in the face of plant 
uncertainty, and moreover, to achieve good performance in the face of this uncer- 
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tainty. In other words, a key objective is robustness. 

It is then not surprising that the classical control designer is comfortable work- 
ing with transfer functions, poles and zeros, magnitude and phase frequency re- 
sponses, and the like. 

The plant models of classical control are linear and of low order. This is the 
case even when the real plant is obviously highly complex and nonlinear. A small 
signal analysis or identification procedure is perhaps the first step to achieve the 
linear models. With such models, controller design is then fairly straightforward. 
For a recent reference, see Ogata (1990). 

The limitation of classical control is that it is fundamentally a design approach 
for a single-input, single-output plant working in the neighborhood of a single op- 
erating point. Of course, much effort has gone into handling multivariable plants 
by closing control loops one at a time, but what is the best sequence for this? 

In our integrated approach to controller design, we would like to tackle con- 
trol problems with the strengths of the frequency domain, and work with transfer 
functions where possible. We would like to achieve high performance in the face 
of uncertainty. The important point for us here is that we do not design frequency 
shaping filters in the first instance for the control loop, as in classical designs, 
but rather for formulating performance objectives. The optimal multivariable and 
adaptive methods then systematically achieve controllers which incorporate the 
frequency shaping insights of the classical control designer, and thereby the ap- 
propriate frequency shaped filters for the control loop. 

Optimal Control 

The strength of optimal control is that powerful numerical algorithms can be im- 
plemented off-line to design controllers to optimize certain performance objec- 
tives. The optimization is formulated and achieved in the time domain. However, 
in the case of time-invariant systems, it is often feasible to formulate an equiva- 
lent optimization problem in the frequency domain. The optimization can be for 
multivariable plants and controllers. 

One particular class of optimal control problems which has proved powerful 
and now ubiquitous is the so-called linear quadratic Gaussian (LQG) method, see 
Anderson and Moore (1989), and Kwakernaak and Sivan (1972). A key result is 
the Separation Theorem which allows decomposition of an optimal control prob- 
lem for linear plants with Gaussian noise disturbances and quadratic indices into 
two subproblems. First, the optimal control of linear plants is addressed assuming 
knowledge of the internal variables (states). It turns out that the optimal solu- 
tions for a noise free ( deterministic ) setting and an additive white Gaussian plant 
driving noise setting are identical. The second task addressed is the estimation 
of the plant model’s internal variables (states) from the plant measurements in a 
noise ( stochastic ) setting. The Separation Theorem then tells us that the best de- 
sign approach is to apply the Certainty Equivalence Principle, namely to use the 
state estimates in lieu of the actual states in the feedback control law. Remarkably, 
under the relevant assumptions, optimality is achieved. This task decomposition 
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allows the designer to focus on the effectiveness of actuators and sensors sepa- 
rately, and indeed to address areas of weakness one at a time. Certainly, if a state 
feedback design does not deliver performance, then how can any output feedback 
controller? If a state estimator achieves poor state estimates, how can internal 
variables be controlled effectively? Unfortunately, this Separation Principle does 
not apply for general nonlinear plants, although such a principle does apply when 
working with so-called information states instead of state estimates. Information 
states are really the totality of knowledge about the plant states embedded in the 
plant observations. 

Of course, in replacing states by state estimates there is some loss. It turns out 
that there can be severe loss of robustness to phase uncertainties. However, this 
loss can be recovered, at least to some extent, at the expense of optimality of the 
original performance index, by a technique known as loop recovery in which the 
feedback system sensitivity properties for state feedback are recovered in the case 
of state estimate feedback. This is achieved by working with colored fictitious 
noise in the nominal plant model, representing plant uncertainty in the vicinity of 
the so-called cross-over frequency where loop gains are near unity. There can be 
“total” sensitivity recovery in the case of minimum phase plants. 

There are other optimal methods which are in some sense a more sophisticated 
generalization of the LQG methods, and are potentially more powerful. They go 
by such names as H <*, and l\ optimal control. These methods in effect do not 
perform the optimization over only one set of input disturbances but rather the 
optimization is performed over an entire class of input disturbances. This gives 
rise to a so-called worst case control strategy and is often referred to as robust 
controller design, see for example. Green and Limebeer (1994), and Morari and 
Zafiriou (1989). 

The inherent weakness of the optimization approach is that although it allows 
incorporation of a class of robustness measures in a performance index, it is not 
clear how to best incorporate all the robustness requirements of interest into the 
performance objectives. This is where classical control concepts come to the res- 
cue, such as in the loop recovery ideas mentioned above, or in appending other 
frequency shaping filters to the nominal model. The designer should expect a trial- 
and-error process so as to gain a feel for the particular problem in terms of the 
trade-offs between performance for a nominal plant, and robustness of the con- 
troller design in the face of plant uncertainties. Thinking should take place both 
in the frequency domain and the time domain, keeping in mind the objectives of 
robustness and performance. Of course, any trial-and-error experiment should be 
executed with the most advanced mathematical and software tools available and 
not in an ad hoc manner. 

Adaptive Control 

The usual setting for adaptive control is that of low order single-input, single- 
output plants as for classical design. There are usually half a dozen or so pa- 
rameters to adjust on-line requiring some kind of gradient search procedure, see 
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for example Goodwin and Sin (1984) and Mareels and Polderman (1996). This 
setting is just as limited as that for classical control. Of course, there are cases 
where tens of parameters can be adapted on-line, including cases for multivari- 
able plants, but such situations must be tackled with great caution. The more 
parameters to learn, the slower the learning rate. The more inputs and outputs, 
the more problems can arise concerning uniqueness of parameterization. Usually, 
the so-called input/output representations are used in adaptive control, but these 
are notoriously sensitive to parameter variations as model order increases. Finally, 
naively designed adaptive schemes can let you down, even catastrophically. 

So then, what are the strengths of adaptive control, and when can it be used to 
advantage? Our position is that taken by some of the very first adaptive control 
designers, namely that adaptive schemes should be designed to augment robust 
off-line-designed controllers. The idea is that for a prescribed range of plant vari- 
ations or uncertainties, the adaptive scheme should only improve performance 
over that of the robust controller. Beyond this range, the adaptive scheme may do 
well with enough freedom built into it, but it may cause instability. Our approach 
is to eliminate risk of failure, by avoiding too difficult a design task or using either 
a too simple or too complicated adaptive scheme. Any adaptive scheme should be 
a reasonably simple one involving only a few adaptive gains so that adaptations 
can be rapid. It should fail softly as it approaches its limits, and these limits should 
be known in advance of application. 

With such adaptive controller augmentations for robust controllers, it makes 
sense for the robust controller to focus on stability objectives over the known 
range of possible plant variations and uncertainties, and for the adaptive or self- 
tuning scheme to beef up performance for any particular situation or setting. In 
this way performance can be achieved along with robustness without the compro- 
mises usually expected in the absence of adaptations or on-line calculations. 

A key issue in adaptive schemes is that of control signal excitation for associ- 
ated on-line identification or parameter adjustment. The terms sufficiently exciting 
and persistence of excitation are used to describe signals in the adaptation context. 
Learning objectives are in conflict with control objectives, so that there must be a 
balance in applying excitation signals to achieve a stable, robust, and indeed high 
performance adaptive controller. This balancing of conflicting interests is termed 
dual control. 



1.3 Robustness and Performance 

With the lofty goal of achieving high performance in the face of disturbances, 
plant variations and uncertainties, how do we proceed? It is crucial in any con- 
troller design approach to first formulate a plant model, characterize uncertain- 
ties and disturbances, and quantify measures of performance. This is a starting 
point. The best next step is open to debate. Our approach is to work with the class 
of stabilizing controllers for a nominal plant model, search within this class for 
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FIGURE 3.1. Nominal plant, robust stabilizing controller 



a robust controller which stabilizes the plant in the face of its uncertainties and 
variations, and then tune the controller on-line to enhance controller performance, 
moment by moment, adapting to the real world situation. The adaptation may in- 
clude reidentification of the plant, it may reshape the nominal plant, requantify 
the uncertainties and disturbances and even shift the performance objectives. 

The situation is depicted in Figures 3.1 and 3.2. In Figure 3.1, the real world 
plant is viewed as consisting of a nominal plant and unmodeled dynamics driven 
by a control input and disturbances. There are sensor outputs which in turn feed 
into a feedback controller driven also by commands. It should be both stabilizing 
for the nominal plant and robust in that it copes with the unmodeled dynamics 
and disturbances. In Figure 3.2 there is a further feedback control loop around the 
real world plant/robust controller scheme of Figure 3.1. The additional controller 
is termed a performance enhancement controller. 



Nominal Plant Models 

Our interest is in dynamical systems, as opposed to static ones. Often for main- 
taining a steady state situation with small control actions, real world plants can be 
approximated by linear dynamical systems. A useful generalization is to include 
random disturbances in the model so that they become linear dynamical stochas- 
tic systems. The simplest form of disturbance is linearly filtered white, zero mean, 
Gaussian noise. Control theory is most developed for such deterministic or sto- 
chastic plant models, and more so for the case of time-invariant systems. We build 
as much of our theory as possible for linear, time-invariant, finite-dimensional dy- 
namical systems with the view to subsequent generalizations. 

Control theory can be developed for either continuous-time ( analog ) models , 
or discrete-time (digital) models , and indeed some operator formulations do not 
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FIGURE 3.2. Performance enhancement controller 

distinguish between the two. We select a discrete-time setting with the view to 
computer implementation of controllers. Of course, most real world engineering 
plants are in continuous time, but since analog-to-digital and digital-to-analog 
conversion are part and parcel of modem controllers, the discrete-time setting 
seems to us the one of most interest. We touch on sampling rate selection, inter- 
sample behavior and related issues when dealing with implementation aspects. 

Most of our theoretical developments, even for the adaptive control loops, are 
carried out in a multivariable setting, that is, the signals are vectors. 

Of course, the class of nominal plants for design purposes may be restricted as 
just discussed, but the expectation in so-called robust controller design is that the 
controller designed for the nominal plant also copes well with actual plants that 
are “near” in some sense to the nominal one. To achieve this goal, actual plant 
nonlinearities or uncertainties are often, perhaps crudely, represented as fictitious 
noise disturbances, such as is obtained from filtered white noise introduced into 
a linear system. 

It is important that the plant model also include sensor and actuator dynamics. 
It is also important to append so-called frequency shaping filters to the nominal 
plant with the view to controlling the outputs of these filters, termed derived vari- 
ables or disturbance response variables, see Figure 3.3. This allows us to more 
readily incorporate robustness measures into a performance index. This last point 
is further discussed in the next subsections. 



Unmodeled Dynamics 

A nominal model usually neglects what it cannot conveniently and precisely char- 
acterize about a plant. However, it makes sense to characterize what has been ne- 
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FIGURE 3.3. Plant augmentation with frequency shaped filters 

glected in as convenient a way as possible, albeit loosely. Aerospace models, for 
example, derived from finite element methods are very high in order, and often 
too complicated to work with in a controller design. It is reasonable then at first 
to neglect all modes above the frequency range of expected significant control ac- 
tions. Fortunately in aircraft, such neglected modes are stable, albeit perhaps very 
lightly damped in flexible wing aircraft. It is absolutely vital that these modes not 
be excited by control actions that could arise from controller designs synthesized 
from studies with low order models. The neglected dynamics introduce phase 
uncertainty in the low order model as frequency increases, and this fact should 
somehow be taken into account. Such uncertainties are referred to as unmodeled 
dynamics. 

Performance Measures and Constraints 

In an airplane flying in turbulence, wing root stress should be minimized along 
with other variables. But there is no sensor that measures this stress. It must be es- 
timated from sensor measurements such as pitch measurements and accelerome- 
ters, and knowledge of the aircraft dynamics (kinematics and aerodynamics). This 
example illustrates that performance measures may involve internal (state) vari- 
ables. Actually, it is often worthwhile to work with filtered versions of these state 
variables, and indeed with filtered control variables, and filtered output variables, 
since we may be interested in their behavior only in certain frequency bands. As 
already noted, we term all these relevant variables derived variables or distur- 
bance response variables. Usually, there must be a compromise between control 
energy and performance in terms of these derived variables. Derived variables are 
usually generated by appropriate frequency shaping filter augmentations to a “first 
cut” plant model, as depicted in Figure 3.3. The resulting model is the nominal 
model of interest for controller design purposes. 

In control theory, performance measures are usually designed for a regulation 
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situation, or for a tracking situation. In regulation, ideally there should be a steady 
state situation, and if there is perturbance from this by external disturbances, then 
we would like to regulate to zero any disturbance response in the derived vari- 
ables. The disturbance can be random such as when wind gusts impinge on an 
antenna, or deterministic such as when there is eccentricity in a disk drive system 
giving rise to periodic disturbances. 

In tracking situations, there is some desired trajectory which should be followed 
by certain plant variables; again these can be derived variables rather than sensor 
measurements. Clearly, regulation is a special case of tracking. 

In this text we consider first traditional performance measures for nominal plant 
models, such as are used in linear quadratic Gaussian (LQG) control theory, see 
Anderson and Moore (1989) and in the so-called control and l\ control the- 
ories, see Francis (1987), Vidyasagar (1986) and Green and Limebeer (1994). 

The LQG theory is derived based on penalizing control energy and plant energy 
of internal variables, termed states, in the presence of white noise disturbances. 
That is, there is a sum of squares index which is optimized over all control actions. 
In the linear quadratic Gaussian context it turns out that the optimal control signal 
is given from a feedback control law. The H 0 0 theory is based on penalizing a 
sum of squares index in the presence of worst case disturbances in an appropri- 
ate class. The l \ theory is based on penalizing the worst peak in the response of 
internal states and/or control effort in the presence of worst case bounded distur- 
bances. Such a theory is most appropriate when there are hard constraints on the 
control signals or states. 

The Class of Stabilizing Controllers 

Crucial to our technical approach is a characterization of the class of all stabilizing 
controllers in terms of a parameter termed Q. In fact, Q is not a parameter such 
as a gain or time constant, but is a stable (bounded-input, bounded-output) filter 
built into a stabilizing controller in a manner to be described in some detail as 
we proceed. This theory has been developed in a discrete-time setting by Kucera 
(1979) and in a continuous-time setting by Youla, Bongiorno and Jabr (1976a). 
Moreover, all the relevant input/output operators, (matrix transfer functions) of the 
associated closed-loop system turn out to be linear, or more precisely affine, in the 
operator (matrix transfer function) Q. In ton, this facilitates optimization over 
stable Q, or equivalently, over the class of stabilizing controllers for a nominal 
plant, see Vidyasagar (1985) and Boyd and Barratt (1991). 

Our performance enhancement techniques work with finite-dimensional adap- 
tive filters Q with parameters adjusted on line so as to minimize a sum of squares 
performance index. The effectiveness of this arrangement seems to depend on the 
initial stabilizing controller being a robust controller, probably because it exploits 
effectively all a priori knowledge of the plant. 

A dual concept is the class of all plants stabilized by a given stabilizing con- 
troller which is parameterized in terms of stable “filter” S. 

In our approach, depicted in Figure 3.4, the unmodeled dynamics of a plant 
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FIGURE 3.4. Plant/controller (Q, S) parameterization 



model can be represented in terms of this “filter” S, which is zero when the plant 
model is a precise representation of the plant. Indeed with a plant parameterized 
in terms of S and a controller parameterized in terms of Q , the resulting closed- 
loop system turns out to be stable if and only if the nominal controller (with 
Q = 0) stabilizes the nominal plant (with 5 = 0) and Q stabilizes S, irrespective 
of whether or not Q and S are stable themselves, see Figure 3.5. This result is a 
basis for our controller performance (and robustness) analysis. With the original 
stabilizing controller being robust, it appears that although S is of high order, 
it can be approximated in many of our design examples as a low order, or at 
least a low gain system without too much loss. When this situation occurs, the 
proposed controllers involving only low order, possibly adaptive Q can achieve 
high performance enhancement. 




FIGURE 3.5. Two loops 



be stabilizing 
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Adaptations 

Adaptive schemes perform best when as much as possible a priori information 
about the plant is incorporated into the design of the adaptive algorithms. It is 
clear that a good way to achieve this is to first include such information into the 
off-line design of a fixed robust controller. 

In direct adaptive control, the parameters for adaptation are those of a fixed 
structure controller, whereas in indirect adaptive control, the parameters for tun- 
ing are, in the first instance, those of some plant model. These plant parameters 
which are estimated on-line, are then used in a controller design law to construct 
on-line a controller with adaptive properties. 

Adaptive schemes that work effectively in changing environments usually re- 
quire suitably strong and rich excitation of control signals. Such excitation of itself 
is in fact against the control objectives. Of course, one could argue that if the plant 
is performing well at some time, there is no need for adaptation or of excitation 
signals. However the worst case scenario in such a situation is that the adjustable 
parameters adjust themselves on the basis of insufficient data and drift to where 
they cause instability. The instability may be just a burst, for the excitation asso- 
ciated with the instability could lead to appropriate adjustment of parameters to 
achieve stability. But then again, the instability may not lead to suitably rich sig- 
nals for adequate learning and thus adequate control. This point is taken up again 
below. Such situations are completely avoided in our approach, because a pri- 
ori constraints are set on the range of allowable adjustment, based on an a priori 
stability analysis. 

One method to achieve a sufficiently rich excitation signal is to introduce ran- 
dom (bounded) noise, that is stochastic excitation, for then even the most “devi- 
ous” adaptive controller will not cancel out the unpredictable components of this 
noise, as it could perhaps for predictable deterministic signals. 

Should there be instability, then it is important that one does not rely on the 
signal build up itself as a source of excitation, since this will usually reflect only 
one unstable mode which dominates all other excitation, and allow estimation of 
only the one or two parameters associated with this mode, with other parameters 
perhaps drifting. It is important that the frequency rich (possibly random) exci- 
tation signal grows according to the instability. Only in this way can all modes 
be identified at similar rates and incipient instability be nipped in the bud by the 
consequent adaptive control action. 

How does one analyze an adaptive scheme for performance? Our approach is 
to use averaging analysis, see Sanders and Verhulst (1985), Anderson, Bitmead, 
Johnson, Kokotovic, Kosut, Mareels, Praly and Riedle (1986), Mareels and Pol- 
derman (1996) and Solo and Kong (1995). This analysis looks at averaging out 
the effects of fast dynamics in the closed loop so as to highlight the effect of 
the relative slow adaptation process. This time scale separation approach is very 
powerful when adaptations are relatively slow compared to the dynamics of the 
system. Averaging techniques tell us that our adaptations can help performance 
of a robust controller. There is less guaranteed performance enhancement at the 
margins of robust controller stability. 
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Iterated Design 

Of course, on-line adaptations using simple adaptive schemes are ideal when these 
work effectively, but perhaps as a result of implementing a possibly crude con- 
troller with limited a priori knowledge, direct or indirect adaptations may not be 
effective. The a priori knowledge should be updated using some identification 
in closed loop, not necessarily of the plant itself. The identification can then be 
used to refine the controller design. This interaction between identification and 
controller design, which is at the heart of on-line adaptive control, is sometimes 
best carried out in an iterative fashion based on a more complete data analysis 
than could be achieved by simple recursive schemes, see for example Anderson 
and Kosut (1991), Zang, Bitmead and Gevers (1991), Schrama (1992a) and Lee, 
Anderson, Kosut and Mareels (1993). Even so, we see such an off-line design 
approach as being in the spirit of adaptive control. Most importantly, we show 
that for a range of control objectives, one can proceed in an iterated identification 
control design manner, without losing the ability to reach an optimal design. That 
is, the iterated control design is able to incrementally improve the control perfor- 
mance at each iteration, given accurate models. Moreover, it can recover from any 
bad design at a later stage. 

Nested or Recursive Design 

Concepts related to those behind iterated design are that of a plug-in controller 
augmentation and that of hierarchical design. We are familiar with the classical 
design approach of adding controllers here and there in a complex system to en- 
hance performance as experience with the system grows. Of course, the catch 
is that the most recent plug-in controller addition may counter earlier controller 
actions. Can we somehow proceed in a systematic manner? 

Also, we are familiar with different control loops dealing with different aspects 
of the control task. The inner loop is for tight tracking, say, and an outer loop is for 
the determination of some desired (perhaps optimal) trajectory, and even further 
control levels may exist in a hierarchy to set control tasks at a more strategic 
level. Is there a natural way to embed control loops within control loops, and set 
up hierarchies of control? 

Our position on plug-in controllers is that controller designs can be performed 
so that plug-in additions can take place in a systematic manner. With each ad- 
dition, there is an optimization which takes the design a step in the direction of 
the overall goal. The optimizations for each introduced plug-in controller must 
not conflict with each other. It may be that the first controller focuses on one 
frequency band, and later controllers introduce focus on other bands as more in- 
formation becomes available. It may be that the first controller is designed for 
robustness and the second to enhance performance for a particular setting, and a 
third “controller” is designed to switch in appropriate controllers as required. It 
may be that the first controller uses a frequency domain criterion for optimization 
and a second controller works in the time domain. Our experience is that a natural 
way to proceed is from a robust design, to adaptations for performance enhance- 
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ment, and then to learning so as to build up a data base of experience for future 
decisions. 

There is a key to a systematic approach to iterated design, plug-in controller 
design, and hierarchical control, which we term recursive controller or nested 
controller design. It exploits the mathematical concept of successive approxima- 
tion by continued linear fraction expansions, for the control setting. 



1.4 Implementation Aspects and Case Studies 

Control theory has in the past been well in advance of practical implementation, 
whereas today the hardware and software technology is available for complex yet 
reliable controller design and implementation. Now it is possible for quite gen- 
eral purpose application software such as Rlab or commercially available pack- 
ages MATLAB* and Xmath + to facilitate the controller design process, and even 
to implement the resultant controller from within the packages. For implementa- 
tion in microcontroller hardware not supported by these packages, there are other 
software packages such as Matcom which can machine translate the generic algo- 
rithms into more widely supported computer languages such as C and Assembler. 
As the on-line digital technology becomes faster and the calculations are paral- 
lelized, the sampling rates for the signals and the controller algorithm complexity 
can be increased. 

There will always be applications at the limits of technology. With the goal 
of increased system efficiency and thus increased controller performance in all 
operating environments, even simple processes and control tasks will push the 
limits of the technology as well as theory. 

One aim in this book is to set the stage for practical implementation of high per- 
formance controllers. We explore various hardware and software options for the 
control engineer and raise questions which must be addressed in practical imple- 
mentation. Multirate sampling strategies and sampling rate selection are discussed 
along with other issues of getting a controller into action. 

Our aim in presenting controller design laboratory case studies is to show the 
sort of compromises that are made in real world implementations of advanced 
high performance control strategies. 



1.5 Book Outline 

In Chapter 2, a class of linear plant stabilizing controllers for a linear plant model 
is parameterized in terms of a stable filter, denoted Q. This work is based on a 
theory of coprime factorization and linear fractional representations. The idea is 



*MATLAB® is a registered trademark of the MathWorks, Inc. 
^Xmath® is a registered trademark of Integrated Systems, Inc. 
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introduced that any stabilizing controller can be augmented to include a physi- 
cal stable filter Q , and this filter tuned either off-line or on-line to optimize per- 
formance. The notion of the class of stabilizing regulators to absorb classes of 
deterministic disturbances is also developed. 

In Chapter 3, the controller design environment is characterized in terms of 
the uncertainties associated with any plant model, be they signal uncertainties, 
structured or unstructured plant uncertainties. The concept of frequency shaped 
uncertainties is developed as a dual theory to the Q-parameterization theory of 
Chapter 2. In particular, the class of plants stabilized by a given controller is stud- 
ied via an 5-parameterization. The need for plant model identification in certain 
environments is raised. 

In Chapter 4, the notion of off-line optimizing a stabilizing controller design 
to achieve various performance objectives is introduced. One approach is that of 
optimal- Q filter selection. Various performance indices and methods to achieve 
optimality are studied such as those penalizing energy of the tracking error and 
control energy, or penalizing maximum tracking error subject to control limits, or 
penalizing peak spectral response. 

Chapter 5 discusses the interaction between control via the Q-parameterization 
of all stabilizing controllers for a nominal plant model and identification via the 
5-parameterization of all plants stabilized by a controller. Two different schemes 
are presented. They differ in the way the identification proceeds. In the so-called 
iterated design, the same 5 parameterization is refined in recursive steps, followed 
by a control update step. In the so-called nested design, successive 5 parameters 
of the residual plant-model mismatch are identified. Each nested 5 parameter has 
a corresponding nested Q plug-in controller. Various control objectives are dis- 
cussed. It is shown that the iterated and nested (Q, 5) design framework is capable 
of achieving optimal control performance in a staged way. 

In Chapter 6, a direct adaptive- Q method is presented. The premise is that the 
plant dynamics are well known but that the control performance of the nominal 
controller needs refinement. The adaptive method is capable of achieving opti- 
mal performance. It is shown that under very reasonable conditions the adaptive 
scheme improves the performance of the nominal controller. Particular control 
objectives we pay attention to are disturbance rejection and (unknown) reference 
tracking. The main difference from classical adaptive methods is that we assume 
from the outset that a stabilizing controller is available. The adaptive mechanism 
is only included for performance improvement. The adaptively controlled loop is 
analyzed using averaging techniques, exploiting the observation that the adapta- 
tion proceeds slowly as compared to the actual plant dynamics. 

The direct adaptive scheme adjusts only a plug-in controller Q. This scheme 
can not handle significant model mismatch. To overcome this problem an indirect 
adaptive- Q method is introduced in Chapter 7. This method is an adaptive version 
of the nested (Q, 5) design framework. An estimate for 5 is obtained on line. On 
the basis of this estimate we compute Q. The analysis is again performed using 
time scale separation ideas. The necessary tools for this are concisely developed 
in Appendix C. 
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In Chapter 8, the direct adaptive- Q scheme is applied for optimal control of 
nonlinear systems by means of linearization techniques. The idea is that in the real 
world setting these closed-loop controllers should achieve as close as possible the 
performance of an optimal open-loop control for the nominal plant. The concept 
of a learning- Q scheme is developed. 

In Chapter 9, real-time controller implementation aspects are discussed, includ- 
ing the various hardware and software options for a controller designer. The role 
of the ubiquitous personal computer, digital signal processing chip and microcon- 
trollers is discussed, along with the high level design and simulation languages 
and low level implementation languages. 

In Chapter 10, some laboratory case studies are introduced. First, a disk drive 
control system is studied where sampling rates are high and compromises must 
be made on the complexity of the controller applied. Next, the control of a heat 
exchanger is studied. Since speed is not a critical factor, sampling rates can be low 
and the control algorithm design can be quite sophisticated. In a third simulation 
study, we apply the adaptive techniques developed to the model of a current com- 
mercial aircraft and show the potential for performance enhancement of a flight 
control system. 

Finally, in the appendices, background results in linear algebra, probability 
theory and averaging theory are summarized briefly, and some useful computer 
programs are included. 



1.6 Study Guide 

The most recent and most fascinating results in the book are those of the last chap- 
ters concerning adaptive- Q schemes. Some readers with a graduate level back- 
ground in control theory can go straight to these chapters. Other readers will need 
to build up to this material chapter by chapter. Certainly, the theory of robust lin- 
ear control and adaptive control is not fully developed in the earlier chapters, since 
this is adequately covered elsewhere, but only the very relevant results summa- 
rized in the form of a user’s guide. 

Thus it is that advanced students could cover the book in a one semester course, 
whereas beginning graduate students may require longer, particularly if they wish 
to master robust and optimal control theory from other sources as well. Also, as 
an aid to the beginning student, some of the more technical sections are starred to 
indicate that the material may be omitted on first reading. 



1 .7 Main Points of Chapter 

High performance control in the real world is our agenda. It goes beyond clas- 
sical control, optimal control, robust control and adaptive control by blending 
the strengths of each. With today’s software and hardware capabilities, there is 
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a chance to realize significant performance gains using the tools of high perfor- 
mance control. 



1.8 Notes and References 

For a modem textbook treatment of classical control, we recommend 
Ogata (1990) and also Doyle, Francis and Tannenbaum (1992). A development 
of linear quadratic Gaussian control is given in Anderson and Moore (1989), and 
Kwakernaak and Sivan (1972). Robust control methods are studied in Green and 
Limebeer (1994) and Morari and Zafiriou (1989). For controller designs based on 
a factorization approach and optimizing over the class of stabilizing controllers, 
see Boyd and Barratt (1991) and Vidyasagar (1985). Adaptive control methods 
are studied in Mareels and Polderman (1996), Goodwin and Sin (1984) and An- 
derson et al. (1986). References to seminal material for the text, found only in 
papers, are given in the relevant chapters. 




CHAPTER 2 



Stabilizing Controllers 



2.1 Introduction 

In this chapter, our focus is on plant and controller descriptions. The minimum 
requirement is that any practical controller stabilizes or maintains the stability of 
the plant. We set the stage with various mathematical representations, also termed 
models, for the plants to be controlled. Our prime focus is on discrete-time, linear, 
finite-dimensional, dynamical system representations in terms of state space equa- 
tions and (matrix) transfer functions; actually, virtually all of the results carry over 
to continuous time, and indeed time- varying systems as discussed in Chapter 8. A 
block partition notation for the system representations is introduced which allows 
for ready association of the transfer function with the state space description of 
the plant. It proves convenient to develop dexterity with the block partition nota- 
tion. Manipulations such as concatenation, inverse, and feedback interconnection 
of systems are easily expressed using this formalism. Controllers are considered 
with the same dynamical system representations. 

The key result in this chapter is the derivation of the class of all stabilizing 
linear controllers for a linear, time-invariant plant model. We show that all stabi- 
lizing controllers for the plant can be synthesized by conveniently parameterized 
augmentations to any stabilizing controller, called a nominal controller. The aug- 
mentations are parameterized by an arbitrary stable filter which we denote by Q. 
The class of all stabilizing linear controllers for the plant is generated as the sta- 
ble (matrix) transfer function of the filter Q spans the class of all stable (matrix) 
transfer functions. 

We next view any stabilizing controller as providing control action from two 
sources; the original stabilizing controller and the controller augmentations in- 
cluding the stable filter Q. This allows us to think of controller designs in two 
stages. The first stage is to design a nominal stabilizing controller, most probably 
from a nominal plant model. This controller needs only to achieve certain limited 
objectives such as robust stability, in that not only is the nominal plant stabilized 
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by the controller, but all plants in a suitably large neighborhood of the nominal 
plant are also stabilized. This is followed by a second stage design, which aims to 
enhance the performance of the nominal controller in any particular environment. 
This is achieved with augmentations including a stable filter Q. This second stage 
could result in an on-line adaptation. 

At center stage for the generation of the class of all stabilizing controllers for a 
plant are coprime matrix fraction descriptions of a dynamical system. These are 
introduced, and we should note that their nonuniqueness is a key in our develop- 
ment. 

In the next section, we introduce a nominal plant model description with focus 
on a discrete-time linear model, possibly derived from a continuous-time plant, as 
for example is discussed in Chapter 8. The block partition notation is introduced. 
In Section 2.3, a definition of stability is introduced and the notion a stabilizing 
feedback controller is developed to include also controllers involving feedforward 
control. In Section 2.4, coprime factorizations, and the associated Bezout identity 
are studied. A method to obtain coprime factors via stabilizing feedback control 
design is also presented. In Section 2.5, the theory for the class of all stabilizing 
controllers, parameterized in terms of a stable Q filter, is developed. The case 
of two-degree-of-freedom controllers is covered by the theory. Finally, as in all 
chapters, the chapter concludes with notes and references on the chapter topics. 



2.2 The Nominal Plant Model 

In this section, we describe the various plant representations that are used through- 
out the book as a basis for the design of controllers. A feature of our presentation 
here is the block partition notation used to represent linear systems. The various 
elementary operations using the block partition notation are described. 



The Plant Model 

The control of a plant begins with a modeling of the particular physical process. 
The models can take various forms. They can range from a simple mathemati- 
cal model parameterized by a gain and a rise time, used often in the design of 
simple classical controllers for industrial processes, to sophisticated nonlinear, 
time-varying, partial differential equation models as used for example in fluid 
flow models. In this book, we are not concerned with the physical processes and 
the derivation of the mathematical models for these. This aspect can be found in 
many excellent books and papers, and the readers are referred to Ogata (1990), 
Astrom and Wittenmark (1984), Ljung (1987) and their references for a detailed 
exposition. 

We start with a mathematical description of the plant. We lump the underlying 
dynamical processes, sensors and actuators together and assume that the mathe- 
matical model includes the modeling of all sensors and actuators for the plant. Of 
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course the range of control signals for which the models are relevant should be 
kept in mind in the design. Actuators have limits on signal magnitudes as well as 
on rates of change of the control signal. What we have termed the plant is depicted 
in Figure 2. 1 . It is drawn as a two-by-two block with input variables w, u and out- 
put variables e, y which are functions of time and are vectors in general. Here the 
block P is an operator mapping the generalized input ( w,u ) to the generalized 
output (e, y). At this point in our development, the precise nature of the operator 
and the signal spaces linked by it are not crucial. More complete descriptions are 
introduced as we progress. Figure 2.1, in operator notation, is then 

EMr} £]• (2i> 

For these models, u is a variable subject to our control, and is termed the control 
input, and u) is a variable not available for control purposes and often consists of 
disturbances and/or driving signals termed simply disturbances. This input w is 
sometimes referred to as an exogenous input or an auxiliary input. The output 
variable y is the collection of measurements taken from the sensors. The two- 
by-two block structure of Figure 2.1 includes an additional disturbance response 
vector e, also referred to as a derived variable. This vector e is useful in assessing 
performance. In selecting control signals u, we seek to minimize e in some sense. 
This response e is not necessarily measured, or measurable, since it may include 
internal variables of the plant dynamics. The disturbance response e will normally 
include a list of all the critical signals in the plant. Note that the commonly used 
arrangement of Figure 2.2 is a special case of Figure 2.1. 




FIGURE 2.2. A useful plant model 
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( 2 . 2 ) 



Note that e = [ «' y' ]' in this specialization, and w\ and u>2 are both disturbance 
input vectors. Note that P22 = G. 



Discrete-time Linear Model 

The essential building block in our plant description is a multiple-input, multiple- 
output (MIMO) operator. In the case of linear, time invariant, discrete-time sys- 
tems, denoted by W in operator notation, we use the following state space de- 
scription: 



W : x k +i = Ax k + Buy, x 0 , ^ 

yk = Cxk + Duk- 

Here k e + = {0, 1, 2, . . . } indicates sample time, x* £ " is a state vector 

with initial value xq, Uk £ p is the input vector and yk e m is the output 
vector. The coefficients, A e nxn , B e nxp , C e mxn and D e mxp are 
constant matrices. For reference material on matrices, see Appendix A, and on 
linear dynamical systems, see Appendix B. 

It is usual to assume that the pair (A , B) is controllable, or at least stabilizable, 
and that the pair (C, A) is observable or at least is detectable, see definitions in 
Appendix B. In systems which are not stabilizable or detectable, there are un- 
stable modes which are not controllable and/or observable. In practice, lack of 
controllability or observability could indicate the need for more actuators and 
sensors, respectively. Of particular concern is the case of unstable modes or per- 
haps lightly damped modes that are uncontrollable or unobservable, since these 
can significantly affect performance. Without loss of generality we assume B to 
be of full column rank and C full row rank*. In our discussion we will only work 
with systems showing the properties of detectability and stabilizability. 

The transfer matrix function of the block W, can be written as 

W(z) = C{zl — A)~ l B + D. (2.4) 

Here W ( 00 ) = D and W (z) is by definition a proper transfer function matrix. 
Thus W € R p , the class of rational proper transfer function matrices. When the 
coefficient D is a zero matrix, there is no direct feedthrough in the plant. In this 
case, W ( 00 ) = 0 and so we have W e R sp , the class of rational strictly proper 
transfer function matrices. 

The transformation from the state space description to the matrix transfer func- 
tion description is unique and is given by (2.4). However the transformation from 

*A discussion of redundancy at input actuators or output sensors is outside the scope of this text 
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the transfer function description to the state space description is not unique. The 
various transformations can be found in many references. The readers are referred 
to Kailath (1980), Chen (1984), Wolovich (1977) and Appendix B. 



Block Partition Notation 

Having introduced the basic building block, in the context of linear MIMO sys- 
tems, we now concentrate on how to interconnect different blocks, as suggested 
in Figure 1.1.1. 

For algebraic manipulations of systems described by matrix transfer functions 
or sets of state-space equations, it proves convenient to work with a block par- 
titioned notation which can be easily related to their (matrix) transfer functions 
as well as their state space realizations. In this subsection, we present the nota- 
tion and introduce some elementary operations. For definitions of controllability, 
observability, asymptotic stability and coordinate basis transformation for linear 
systems, the readers are referred to Appendix B. 



Representation 

Let us consider a dynamic system W with state space description given in (2.3). 
The (matrix) transfer function is given by (2.4) and in the block partition notation, 
the system with state equations (2.3) is written as 



W : 



A B 
C D 



(2.5) 



The two solid lines within the square brackets are used to demarcate the 
(A, B, C, D) matrices, or more generally the (1, 1), (1, 2), (2, 1) and (2, 2) sub- 
blocks. In this notation, the number of input and output variables are given by 
the number of columns and rows of the (2, 2) subblock, respectively. In the case 
where (A, B) is controllable and (A, C) is observable, the representation of the 
system (2.5) is said to be a minimal representation and the dimension of the (1,1) 
subblock gives the order of the system. 



Subpartitioning 



With block partitioning of A, B, C, D as in the following state equations, 



'An 


A12" 


Xk + 


'fin 


Bnl 


_A 2 i 


a 22 _ 


B 2 1 


Bn J 


[Cii 


Ci 2 1 


Xk + | 


~D n 


£>12] 


|_C 2 1 


C 22 J 


D 2 1 


£>22 J 



(2.6) 
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the block partition notation for the system W is 





' An 


A 12 


Bn 


B\2 


W12] . 


A21 


A 22 


B 2 \ 


B22 


W22 ] ' 


Cn 


C12 


Du 


Dn 




Cat 


C22 


D 21 


D 22 _ 



where Wij denotes the state space equations for the subsystem with input j and 
output i. By inspection, 



W n : 


' An 
A21 


A 12 
A22 


Bn ' 
B 2 \ 


W 12 : 


' An 
A21 


A 12 
A22 


B21 ' 
B22 




Cn 


C12 


Dn _ 




Cn 


C12 


D\ 2 




' An 


A12 


Bn ' 




' An 


A 12 


B\2 " 


W21 : 


A21 


A22 


B 2 1 


W22: 


A21 


A 22 


B 2 2 




. C21 


C22 


£>21 _ 




. C21 


C22 


D 2 2 _ 



and the associated state space equations of the subsystems W(j can be written 
down immediately, for example 




y\k — [C|, C12] Xk + DnU2k- 



These state space realizations are not necessarily minimal. 



Sums 

Let us consider the parallel connection of two systems with the same input and 
output dimensions. For two systems, W\ and W2 given as 



' A 2 


B 2 " 


Cl 


Di _ 



" A\ 


Bi " 


. Ci 


D i _ 



(2.9) 



their sum (parallel connection) in block partition notation is given by: 





' A! 


0 


Bi 


Wi + W 2 : 


0 


a 2 


b 2 




. Ci 


c 2 


£>1 + D 2 _ 



( 2 . 10 ) 
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This can be checked by examining the state-space model of the parallel connec- 
tion. The order of the sum of the (matrix) transfer functions is the sum of the 
orders of each transfer function, in general. However in certain cases, as when 
A i — A2, we can eliminate uncontrollable or unobservable modes in the parallel 
connection to achieve a reduced order minimal realization. (See problems at the 
end of the chapter for some discussion.) 

Multiplication 

Consider again two systems as given in ( 2 . 9 ), assuming that the output dimension 
of W2 equals the input dimension of W\ . The product, or series connection, of the 
two systems W\ and W2 (W\ after W2) may be represented as: 





' Ai 


B1C2 


BlD 2 


W1W2 : 


0 


A 2 


B 2 




. Cl 


dic 2 


D\D 2 _ 



Again this representation is not minimal when there are pole/zero cancellations 
in W1W2, although it is minimal in the generic case; that is when A\, A 2, B\. 
B2, Ci, C2, D \ , Dj have no special structure or element values. (Notice that 
W\W2 / W2 W\. Also, even when W1W2 is defined, W2W1, may not be!). 

Inverse 

It is readily checked that under the condition that D j -1 exists, the inverse of the 
system W\ exists and is proper and is given by 



' Ai - B X D~ 1 C X 


~B\D~ l ’ 


D\ X C\ 





Notice that 



W~' W\ = IT, W l 



r _i [0 o’ 
1 ' 0 / 



( 2 . 12 ) 



( 2 . 13 ) 



Transpose 

The transpose of the system W\ is given by 






A 'i 



C'l 



B[ | D\ 



( 2 . 14 ) 
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Stable Uncontrollable Modes 

Consider the following system W with its associated state-space realization: 





" An 


A 12 


Bi " 


W : 


0 


A22 


0 




. Cl 


c 2 


D 



where the eigenvalues of A22 are inside the unit circle, that is A,' (A22H < 1 , 
and so the modes associated with A22 are asymptotically stable. Clearly for the 
system W with its associated state-space realization, the states associated with 
A22 are not controllable from the inputs. Moreover, since the modes associated 
with A22 are asymptotically stable, the associated states decay exponentially to 
zero from arbitrary initial values. From a control point of view these modes may 
be ignored, simplifying the W representation to: 



W s : 



A11 


Bi " 


. C i 


D 



( 2 . 16 ) 



Stable Unobservable Modes 

The “dual” case to the above is where there are stable unobservable modes. Con- 
sider the system with associated state-space realization given by 





" An 


0 


Bi " 


W : 


A21 


A22 


b 2 




. Ci 


0 


D 



where again 22) < 1 for all i. In this case, the states associated with A 22 
can be excited from the inputs but they do not affect the states associated with 
An or the system output. These states are therefore not observable and if ignored 
allow the simpler representation. 



W s : 



An B 1 

Ci D 



( 2 . 18 ) 



The above simplifications are allowed from the control point of view. Obviously 
these unobservable/uncontrollable models do affect the internal system behavior 
and the transient behavior. 



Coordinate Basis Transformation 

For a linear system W of ( 2 . 5 ), let us consider the effect of transformations of 
the associated state-space variables. Thus consider transformations of the state 
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vector x, input vector u and output vector y by nonsingular matrices T, S and R 
as follows. 



x = Tx, u = Su, y = Ry. 
The system from u to y, denoted W, is then given by 



W : 



TAT~ l TBS~ l 
RCT~ X RDS~ l 



(2.19) 



( 2 . 20 ) 



On many occasions in the book, cascade or parallel connections of subsystems 
using formulas given in (2.10) or (2.11) give rise to high dimensional systems W 
with stable uncontrollable or unobservable modes in their state-space realizations. 
The associated system descriptions can then be simplified by removing (or ignor- 
ing) such modes as in deriving W s . That is, it is possible to find a transformation 
matrix, T (with R — I and S = /) such that the transformed (matrix) transfer 
function is of the form (2.15) with stable uncontrollable modes, or (2.17) with 
stable unobservable modes. The stable uncontrollable or unobservable modes can 
then be removed by the appropriate line and column deletion to achieve the sim- 
pler system description W4 

In fact, a number of the manipulations in our controller design approaches yield 
system descriptions that can be simplified by one of the following very simple 
transformations T. 

[4 [:-/] [: a- [-4 - 

There are short cuts when dealing with these simple transformations. The trans- 
formations can be performed using operations similar to the elementary rows and 
columns operations in the solution of a set of linear equations using the Gaussian 
elimination method. Let us work with an example. 

Example. Consider the system W given as 





' An 


A 12 


Bi ' 


W : 


A21 


A22 


b 2 




. Ci 


c 2 


D 



where An, A 12, A21 and A22 are of the same dimension. Under the transforma- 
tions R — /, S = I and 






(2.23) 
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the system W can be represented in the transformed coordinate basis 





An + A 2 i 


Ai 2 + A 22 - An - A 2i 


Bi + B 2 ' 


W : 


A 2 i 


A 22 - A 2 i 


b 2 




Ci 


C 2 -Ci 


D 



The transformed system description is obtained as follows. First ignore the verti- 
cal and horizontal partitioning lines and view IT as a 3 block by 3 block structure. 
Now add block row 2 of the array to block row 1 of the array to get the intermedi- 
ate structure: 



An + A 2 i 


Ai 2 + A 22 


B\ + B 2 


A 2 i 


a 22 


b 2 


Ci 


c 2 


D 



Leaving block column 1 of the above intermediate array unchanged, we then sub- 
tract block column 1 from the block column 2 to get the final transformed system 
description of (2.24). In fact, we can generalize the procedure as follows. 

For a system description with p inputs and m outputs expressed in the block 
partition form such that the A matrix of the associated state-space realization is an 
(n x n) matrix, then the input/output property of the system W is not changed by 
the following operations. Interpreting the block partition notation representation 
of the system as an (n + m) x (n + p) array, add (subtract) the ith row to the y'th 
row of the array where 1 < i, j < n, leaving the ith row unchanged, to give an 
intermediate array, and then follow by subtracting (adding) the jth column from 
the ith column of the intermediate array, leaving the jth column unchanged. 



Main Points of Section 

In this section, a two port operator description for the plant is introduced. Plant 
transfer function matrix and state space representations are given in the context of 
linear, time-invariant, discrete-time systems. A shorthand block partition notation 
is presented which allows ready manipulation of subsystems, including series and 
parallel connections and inverses. 



2.3 The Stabilizing Controller 

In this section, we first work with a controller and model of the plant in a closed- 
loop feedback control arrangement used for regulation of output variables to zero 
in the presence of disturbances. We then move on to the more general tracking 
case which has feedforward control as well as feedback control so that there is 
close tracking of external signals in the presence of disturbances. 
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The Closed-loop System 

Consider the plant model shown in Figure 2.1. Notice that since P is a linear 
operator it can be block partitioned into four operators as 



P = 



Pu P\2 
fl\ P22_ 



(3.1) 



Let us introduce a feedback controller A" in a feedback control arrangement as 
shown in Figure 3.1. The block P generates not only the plant sensor outputs 
y, but also the signal e, termed the disturbance response, which is used to eval- 
uate the performance of the feedback system. The feedback controller seeks to 
minimize the disturbance response e in some sense. The feedback controller is 
sometimes referred to as a one-degree-of-freedom controller, in contrast to a two- 
degree-of-freedom controller with additional command inputs discussed subse- 
quently. 



Disturbance 



Disturbance response 




FIGURE 3.1. The closed-loop system 



For the system of Figure 3.1, we have the following equations in operator no- 
tation: 



[:]=[:: ztl 

u = Ky. 

Assuming xvell-posedness of the interconnection, or equivalently, that the relevant 
inverses exist, the disturbance response is given by: 



= F k w, 



Fk = Pu + PnKlI - P 2 2K)- l P 2 i. 



(3.3) 
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A Stabilizing Feedback Controller 

Next we consider stability issues. We restrict ourselves in this chapter to the case 
where all operators correspond to time-invariant, multiple-input multiple-output, 
discrete-time, finite-dimensional systems. In this context all operators have cor- 
responding proper rational transfer function matrices. In Chapter 7, the case of 
time- varying systems is seen to follow much of the development here when the 
transfer functions are generalized to time-varying operators. 

The underlying stability concept of interest to us here is bounded-input, 
bounded-output stability (BIBO): 

Definition. A system is called BIBO stable if any norm bounded input yields a 
norm bounded output. When the class of inputs is bounded in an I2 norm sense 
and leads to I2 norm bounded outputs, the system is said to be BIBO stable in an 
I2 sense. (Recall that I2 bounded signals in discrete-time are square summable, 
see also Appendix B.) 

In the context of linear systems, BIBO stability in an £2 sense is equivalent to 
small gain stability and asymptotic stability of the state space description. Trans- 
fer functions which are BIBO stable in an I2 sense are said to belong to the H 0 0 
space and if rational to belong to the rational space, denoted RH^. For fur- 
ther technical details on these spaces and stability issues, see Appendix B. Suffice 
it to say here: H (z) e RHoq if and only if every element of H (z) is rational and 
has no pole in |z| > 1. 

As a starting point, we use the representation as in Figure 3.2 to discuss stabil- 
ity. 




Let us consider the feedback control loop of Figure 3.2 with a feedback con- 
troller K e R p applied to a plant G e R p . Noting that 




then we have under well-posedness, or equivalently assuming that the inverse 
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exists, 



[;]-['« -,‘T'fc] 

Closed-loop stability, also termed internal stability, is defined as BIBO stability 
in an I2 sense, in that any bounded input to the closed-loop system will give 
rise to bounded-output signals everywhere within the loop. Here for linear, time- 
invariant systems, internal stability is identical to asymptotic stability of the state 
space description of the closed loop. Applying these notions to (3.4) leads to the 
following result. 

Theorem 3.1. A necessary and sufficient condition to ensure internal stability of 
the feedback control loop of Figure 3.2 is that 

[4 

Equivalently, 

(/ - KG)~ l 
G(I - KG)~ 

Definition. We say that K stabilizes G or (G, K) is a stabilizing pair if (3.5) or 
equivalently (3.6) holds. 

We see that for K to stabilize G, there is a requirement that each of the four 
(matrix) transfer functions (/ — KG) -1 , K(I — GK)~ l , G(I — KG)~ l and (/ — 
GK)~ l be asymptotically stable. For the single-input, single-output case, where 
G and K are scalar transfer functions, the stability condition (3.6) is equivalent to 
the stability condition that (/ — GK)~ l e RH 0 c , and that there be no pole/zero 
cancellations in the open-loop transfer function KG, or GK. 

For the system of Figure 3.1, internal stability will imply stability of the closed- 
loop (matrix) transfer function Fg of (3.3). Moreover, Theorem 3.1 is readily gen- 
eralized by working with a rearranged version of Figure 3.1, namely Figure 3.3. 
Thus apply the results of Theorem 3.1 with G replaced by P and K replaced by 
^ ]. This leads to the following. 

Theorem 3.2. A necessary and sufficient condition to ensure internal stability of 
the closed-loop feedback control system of Figure 3.1, or equivalently Figure 3.3, 
is that 



■ 



K(I - GK)~ 

c i-gk r 1 



— P 




(3.7) 
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FIGURE 3.3. A rearrangement of Figure 3.1 
or equivalently, with a partitioning of P as in ( 2 . 1 ) 

A"] e RHoo, 

( 3 . 8 ) 

We remark that a necessary condition for stability is that the pair (P22, K) is 
stabilizing. To connect to earlier results we set P22 = G so that this condition is 
the familiar condition that (G, K) is stabilizing. If in addition, P\ \ e RHoq, and 
P21, Pi 2 belong to RH 0 0 or P21 and/or P\2 are identical to G = P22, then this 
condition is also sufficient for stability. 

A Stabilizing Feedforward/Feedback Controller 

Consider the feedforward/feedback controller arrangement of Figure 3 . 4 . Notice 
that in this arrangement, both the plant output y, and a reference signal d, possi- 
bly some desired output trajectory, are used to generate the control signal. Con- 
sider now the rearrangement of the scheme of Figure 3.4 as a feedback controller 
scheme for an augmented plant depicted in Figure 3 . 5 . The original plant G and 
disturbance signal W2 are augmented as follows: 




I" ' -*x « 

L-fti / J 






P i2 (I ~ KP22)- 1 [I 
P22K)~ 1 P 2 1 e RHoo. 



hit = [*/*] 



FIGURE 3.4. Feedforward/feedback controller 
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FIGURE 3.5. Feedforward/feedback controller as a feedback controller for an augmented 



Under this augmentation, the two-degree-of-freedom controller for G is identical 
to the one-degree-of-freedom controller for the augmented plant . Applying the 
earlier internal stability results for a one-degree-of-freedom controller to this aug- 
mented system leads to internal stability results for the two-degree-of-freedom 
system. Thus consider a two-degree-of-freedom controller = [ Kf k ] g R p for 
the plant e R p . Then the controller internally stabilizes and thus if and 
only if 



I - 


- 1 


[/] - 


K f K\ 


I 


= 




1 


L J 




L L g J 


L° J 



or equivalently, if and only if 

(/ — KG)~ l (/ — KG)~ l K f K(I - GK)~ l 
0/0 
\_GiI-KG)- 1 G(I — KG)~ 1 K f (/ — GK)~ l J 

or equivalently, if and only if 

[-0 €RH - 



The first condition tells us that there must be internal stability of the feedback 
system consisting of G and the feedback controller K. The second condition tells 
us that any unstable modes of Kf must be contained in K . That is, in the event 
that Kf is unstable, it should be implemented along with K in the one block = 
[k K f ~\ with a minimal state space representation. If Kf and K are implemented 
separately, then K / must be stable. This is a well known result for the stability 
of two-degrees-of-freedom controllers, see Vidyasagar (1985), but the derivation 
here is more suited to our approach. 
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Main Points of Section 

In this section we have considered stability properties when applying controllers 
to plants. Internal stability is linked to the stability of certain (matrix) transfer 
functions associated with the feedback loops. Both one-degree-of-freedom and 
two-degrees-of-freedom controller structures are examined. Conditions ensuring 
stability are identified. 



2.4 Coprime Factorization 

An important step towards the next section’s objective of characterizing the class 
of all stabilizing controllers is the coprime factorization of the plant model and 
controller. For scalar models and controllers, factorization leads to the models 
and controllers being represented as the ratio of two stable transfer functions. 
This factorization is termed coprime when the two transfer functions have no 
common zeros in |z| > 1. Coprimeness excludes unstable pole/zero cancellations 
in the fractional representation. In the multivariable case, the plant model and 
nominal controller (matrix) transfer functions are factored into the product of a 
stable (matrix) transfer function and a (matrix) transfer function with a stable 
inverse. Coprimeness can be expressed as a full rank condition on the matrices in 
|z| > 1, see below. In this section, we discuss various ways to achieve coprime 
factorizations. 



The Bezout Identity 

Let us denote stable, coprime factorizations for the plant G(z) £ R p of (2.3) and 
a nominal controller K (z) £ R p as follows. 



G = NM~ l = M~ l N\ N,M,N,M £RH OQ , (4.1) 

K = UV~ 1 = V~ l U; U, V, U, V £ RHoo. (4.2) 



It turns out that there exist such factorizations for any plant and controller. We 
defer until later in this section the question of existence and construction of such 
factorizations in our setting. Coprimeness of the factors N and M means that 
[ jy ] has full column rank in |z| > 1, or equivalently, that its left inverse exists in 
RH 0 0 . Coprimeness of M, N requires correspondingly that \m n] has full rank 
in |z| > 1 , or equivalently, has a right inverse in RH 0 G . 

To illustrate the idea, consider the more familiar setting of scalar transfer func- 
tions. Let 



G(z) = 



Hz) 

Hz)’ 



(4.3) 



where b(z), a(z) are polynomials in z. Assume that a has degree n, b is of degree 
less than or equal to n, and that a is monic in that the coefficient of z" is unity. 
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Also, assume that a(z), b{z) have no common zeros. A coprime factorization as 
in (4.1) is then given as 

G(z) = • (4.4) 

By construction b(z)/z n , a{z)/z n e RH 0 Q . 

In our setting here, we restrict attention to the situation where K stabilizes G 
and later give existence and constructive procedures for achieving the desired fac- 
torizations. First, we study in our context a set of equations known as the Bezout 
or Diophantine equations which are associated with coprimeness properties. 

As a first step, examine the four closed-loop (matrix) transfer functions of (3.6), 
under the equalities of (4.1) and (4.2). 

It is straightforward to see that 

(/ - KG ) _1 = (/ - V~ l UNM~ 1 )~ 1 
= M(VM -UN)~ l V, 

G(I - KG)~ l = N(VM - t/A) -1 V, 

Also, since (I - KG)K = K(I - GK), then 

K(I - GK ) _1 = (/ - KG)~ l K 

= M{VM -UN)~ l U, 
and 

(/ - GK)~ X = / + GK(I - GK r 1 
= I + G(I - KG)~ l K 
— I + N(VM — UN)~ l U. 

Thus 

[-„ «]*[::] «*■ 

This result leads to the following lemma. 

Lemma 4.1. Consider the plant G e R p , and controller K e R p . Then K stabi- 
lizes G if and only if there exist coprime factorizations G = AM -1 = M~ l N, 
K = UV~ l = V~ l U with N, M, N, M, U, V, U, V e RHoo such that either of 
the following Bezout (Diophantine) equations hold, 



(4.5) 

(4.6) 

(4.7) 

(4.8) 



VM -UN = /, 
MV -NU = /, 



(4.10) 

(4.11) 




36 Chapter 2. Stabilizing Controllers 

or equivalently, under the lemma conditions the following double Bezout equation 
holds 



r* -fir" u ]J m *ir* -f|=p °i. (4.i2) 

\_-N M J V\ [ N V\ \_-N M J |_° /J 

Proof. It is clear that if the stable pairs M, N and U, V satisfy the Bezout equa- 
tion (4.10), the right hand side of (4.9) is stable. This implies that the left hand 
side of (4.9) is stable, or equivalently, K stabilizes G. Conversely, if K stabilizes 
G, the left hand side of (4.9) is stable. Because M, N and U, V are coprime there 
exists a left inverse for [ ^ ] and a right inverse for [v u] in RH 0 Q . Hence we 
obtain the result that 



(VM - UN ) _1 = Z e RH 0 0 . (4.13) 

Now, define a new coprime factorization for G : G = Then (4.10) 

follows. 

By interchanging the role G and K in the above arguments so that we view 
G as stabilizing K, we are led to the alternative dual condition to (4.10), namely 
(4.11). □ 

Corollary 4.2. With (4.1), (4.2) holding, necessary and sufficient conditions for 
the pair (G, K ) to be stabilizing are that 




These conditions are equivalent to (3.5), (3.6). 

The above corollary is transparent noting that (4.12) implies (4.14), and (4.14) 
leads to 



or equivalently, that (G, K) is a stabilizing pair. 

Normalized Coprime Factors 

Normalized coprime factors ( M , N) or (M, N) are left or right coprime factors 
with the special normalization property 



N~N + M~M - /, 
NN~ + MM~ = /, 



(4.15) 
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for all | z | = 1 . Here M~(z ) denotes M' (z ~ 1 ) etc. Our developments do not feature 
these factorizations with their interesting properties as do those of McFarlane and 
Glover (1989). We note in passing that these factorizations may be obtained in 
principle from any coprime factors by a spectral factorization. Thus for NN~ + 
MM" = ZZ~ with Z, Z _1 e RH 0 0 we have normalized left coprime factors 
(Z~ l N, Z~ l M). Likewise for N~N + M~M = Z~Z with Z, Z~ l e RH a 0 we 
have normalized right coprime factors [NZ~ l , MZ -1 ). 



State Estimate Feedback Controllers 

Let us construct stable, coprime factorizations for a plant model and a special 
class of controllers known as state estimate feedback controllers. Consider a plant 
G with a state space description given as follows. 



G : 



A B 
C D 



(4.16) 



Under a stabilizability assumption on the pair (A, B), it is possible using stan- 
dard methods to construct a constant stabilizing state feedback gain (matrix) F, 
in that (A + BF) has all eigenvalues within the unit circle. Likewise, under a de- 
tectability assumption on the pair (A, C), it is possible to construct a constant sta- 
bilizing output injection (matrix) H , in that (A + HC) has all eigenvalues within 
the unit circle. The gains F and H can be obtained from various methodolo- 
gies for controller designs. Examples are the so-called linear-quadratic-Gaussian 
(LQG) methods, see Anderson and Moore (1989), Kwakernaak and Sivan (1972) 
or eigenvalues assignment approaches in Ogata (1990). The stabilizing controller 
K with its associated state space is then given by 



’ A + BF + HC + HDF 


-H " 


F 


0 



The plant/controller arrangement is depicted in Figure 4.1. Stable coprime fac- 
torizations for G(z.) and K (z) are then given as follows. 



\m t/1 

[n v\ : 


A + BF 


B -H 




F 

_ C + DF 


I 0 

D I 




P -fl: 

| _-N M J 


” A + HC 


\-(B + HD) Hi 


F 

C 


I 

-D 


0 

I 



(4.19) 
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To verify that NM 1 is indeed a factorization for G, proceed as follows: 



M : 



A + BF B 
F I 



’ A + BF 


B 


C + DF 


D 



Here M 1 obviously exists, and has a representation 



" A + BF - BF 


-B ’ 




’ A 


-B " 


F 


I 




F 


/ 



We have then for NM 1 : 





' A + BF 


BF 


B 


NM- 1 : 


0 


A 


-B 




C + DF 


DF 


D 



Using the state space transformation T = [ q \ ] which leaves NM 1 unchanged, 
we find 





' A + BF 


0 


0 


NM- 1 : 


0 


A 


—B 




C + DF 


-C 


D 



Hence after removing stable, uncontrollable modes and changing the signs of the 
input and output matrices, we have: 



A B 
C D 



: G. 



NM~ l : 
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In a similar way, we can show that UV~ l is a factorization of K . 

These factorizations as presented by (4.18), (4.19) satisfy the double Bezout 
equations. This may be verified by a now familiar sequence of steps as follows: 



" A + BF 


B -H 


" A + HC 


-(B + HD) 


H ' 


F 


I 0 


F 


I 


0 


C + DF 


D 1 


C 


-D 


I 



’ A + BF 


BF - HC 


(B + HD) 


—H " 


0 


A + HC 


-(B + HD) 


H 


F 


F 


I 


0 


C + DF 


C + DF 


0 


I 



A + BF 


0 


0 


0 


0 


A + HC 


-(B + HD) 


H 


F 


0 


I 


0 


C + DF 


0 


0 


/ 




Notice that the second equality is obtained via the block state transformation 
T = [//], with identity input and output transformations. The third equality 
is obtained by removing uncontrollable and unobservable modes. 

It turns out that special selections of stabilizing F and H yield the normalized 
coprime factors satisfying (4.16). More on this in Chapter 4. 



More General Stabilizing Controllers ^ 

We consider the coprime factorizations with respect to an arbitrary stabilizing 
controller K. Let K be given by 



K : 



" A 


B " 


C 


D 



(4.20) 



^This material, included for completeness, is somewhat condensed and may be merely skimmed 
on first reading. 
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with the pairs (A, B) stabilizable and (A, C) detectable. Because (G, K) is 
bilizing: 




’ A 


0 


— B 


0 


0 


A 


0 


—B 


0 


c 


/ 


—D 


c 


0 


-D 


I 



’ A + BY DC 


BYC 


BY 


BYD " 


BZC 


A+BZDC 


BZD 


BZ 


YDC 


YC 


Y 


YD 


ZC 


ZDC 


ZD 


Z 



(4.21) 



where Y = (/ - DD)~ l and Z = (/ - DD)~K 
To perform the coprime factorization, a key step which only becomes obvious 
in hindsight is to first construct state feedback gains F and F under stabilizability 
assumptions on the pairs (A, B) and (A, B) such that A + BF and A + BF have 
all eigenvalues within the unit circle. These matrix gains are easily obtained by 
performing a state feedback design for the pairs (A, B) and (A, /}), respectively. 
The coprime factorizations for G and K are then given by 



’ A + BF 


0 


B 


0 


0 


A + BF 


0 


B 


F 


C + DF 


I 


D 


C + DF 


F 


D 


I 



' A + BY DC 


BYC 


-BY 


BYD ' 


BZC 


A + BZDC 


-BZD 


BZ 


F -YDC 


-YC 


Y 


YD 


ZC 


-{P- ZDC^j 


ZD 


Z 



Comparing with the closed-loop (matrix) transfer functions (4.21), it is clear that 
the factorizations here are stable only because of the construction of the state feed- 
back gains F, F. The factorizations can be verified to satisfy the double Bezout 
equation by direct multiplication; see problems. 
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Main Points of Section 

In this section, we have examined the representation of the (matrix) transfer func- 
tion of a plant and its stabilizing controller using stable, coprime factors. A key 
result is that internal stability of the closed-loop system is equivalent to the ex- 
istence of coprime fractional representations for the plant model and controller 
that satisfy the Bezout equation. This has been exploited to provide an explicit 
construction of coprime factorizations of the plant model and a stabilizing state 
feedback controller. Coprime factorizations associated with other stabilizing con- 
trollers can be derived. 



2.5 All Stabilizing Feedback Controllers 

In this section, the class of all stabilizing controllers for a plant is parameterized 
in terms of a stable, proper filter, denoted Q. 



The Q -Parameterization 

Let K & R p be a nominal controller which stabilizes a plant G e R p and with 
coprime factorizations given by (4.1), (4.2) satisfying the double Bezout equation 
(4.12). Consider also the following class of controllers K (Q) parameterized in 



terms of Q € RH 0 0 and in right factored form: 

K(Q):=U(Q)V(Q)-\ (5.1) 

U(Q) = U + MQ, V(Q) = V + NQ ; (5.2) 

or in left factored form: 

K(Q) := V(Q)~ l U(Q), (5.3) 

U(Q) = U + QM, V(Q) = V+QN. (5.4) 



Then, as simple manipulations show, these factorizations together with the fac- 
torizations for G also satisfy a double Bezout equation (4.12), so that K(Q) sta- 
bilizes G for all Q e RH 0 G . Equation (5.2) is referred to as a right stable linear 
fractional representation, while (5.4) is referred to as a left stable linear fractional 
representation. 

We now show the more complete result that with Q spanning the entire class 
of RH 0 o, the entire class of stabilizing controllers for the plant G is generated. 
Equivalently, any stabilizing controller for the plant can be generated by a partic- 
ular stabilizing controller and a Q C RH 0 0 . 

Let us consider the closed-loop system with the controller of (5.2) or (5.4). 
This closed-loop system can be written in terms of the closed-loop system with 
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the nominal controller and the stable transfer function Q as follows. 



' i -Kiefl 1 [ 


I 


~V(Q)~ l U(Q) 


L-o ' J 1 


-M~ l N 


I 



M U (Q) V(Q) 0 



M U V 0 

N V 0 M 



MQN 0 0 MQM 

NQN ol + I 0 NQM 



\v -or 


1 r - 1 


n-l 

0 


\mqn 


MQM 


\_-N M J 


0 


M -1 


[iveiv 


NQM 



(I-K(Q)G)- 1 
G ( I-K(Q)G r 1 
M C/1 r V 0 1 
N V O M 



K (I — K(Q)G)~ 1 
{ I-GK(Q )-!) 



Note that the third and last equalities are obtained by applying the double Bezout 
equation. We conclude that 



From (5.7), it is again clear that any controller K (Q) parameterized by Q e RH^ 
as in (5.2) stabilizes the plant G. To see the converse, we note that Q is given as 
follows. 



From (5.8), we note that if both K and K(Q) stabilize G, that Q as given by (5.8) 
satisfies Q e RH a 0 . 
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Consider now an arbitrary stabilizing controller for G, denoted K\. Replacing 
K(Q ) in (5.8) by K \ allows a calculation of a K\ dependent Q, denoted Q\. and 
associated U\ = U + MQ\, Vi = V + NQ u Ui = U+QiM, Vi = V+QiN. 
Now reversing the derivations for (5.5)— (5.8) in the case K (Q) = K \ leads to the 
first equality in (5.5) holding with K(Q), V(Q), U{Q) replaced by K u V u U u or 
equivalently, K\ = Vf 1 U i (and G = M~ X N), which allows a construction of an 
arbitrary stabilizing controller K\ using our controller structure with Q replaced 
by Q\. Consequently, we have established the following theorem. 

Theorem 5.1. Given a plant G e R p with coprime factorizations (4.1), (4.2), and 
the controller class K{Q) given from (5.2), (5.4), then (5.7), (5.8) hold. Moreover, 
K(Q ) stabilizes G if and only if Q e RH 0 Q . Furthermore, as Q varies over RH 0 0 
all possible proper stabilizing controllers for G are generated by K(Q). 

Proof. See above. □ 



Realizations 

We will next look at the realization of the controller K (Q) parameterized in terms 
of Q e RH 0 0 . From (5.2) and the Bezout equation (4.12), we have 

K(Q) = {U + MQ){V + NQ)~ l 

= (U+ V~\l + UN)Q)(V + NQ)~ l 
= (U + V- 1 Q + V~ 1 UNQ)(V + NQ)- 1 (5.9) 

= ( uv~\v + NQ) + y- 1 2)(y + nq)~ x 
= K + V~ 1 Q(I + V~ 1 NQ)~ l V~ x . 



The controller K(Q ) as given by (5.9) can be organized into a compact form 
depicted in Figure 5.1 with J given as 



and 



J - 



'uv~ x V~ x 1 _ \v~ x u V~ x 

y- 1 -y-^J _ [ y 1 -V~ X N ’ 




(5.10) 



(5.11) 



With the internal structure of the controller K(Q) shown in Figure 5.2, it is 
interesting to note that effectively, the control signal generated by K{Q) consists 
of the signal generated by the nominal controller K and another second control 
loop involving the stable transfer function Q. When Q = 0, then K (Q) = K. 

For implementation purposes, the Bezout equation of (4.12) can be used to 
obtain a variant of the block J in Figure 5.2. The reorganized structure is shown 
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in Figure 5.3. With this structure, and as shown below, the signal r is generated 
by r = My — Nu, where M and N are stable filters. Compare to the structure 
in Figure 5.2, where the signal r is generated using a feedback loop involving 
N, V~ l and Q, with V -1 possibly unstable. The structure of Figure 5.3 is more 
desirable. 

From (5.1 1), we have immediately 



(5.12) 
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FIGURE 5.3. Reorganization of class of all stabilizing controllers 



Using the second equation from (5.11), and (5.12) we have 
r = V~'y - V-'N (vu - Uyj 

= V -1 (i + NU) y - V~ l NVu 

y > (5.13) 

= V 1 (vM^jy- V~ l VNu 

= My — Nu 



It may appear at this stage that to implement the controller K{Q) requires aug- 
mentations to K for the generation of filtered signals r for driving the Q filter. 
This is not the case for the state estimate feedback controller structure, as we now 
show. 

In the state space notation of (4.17)-(4.19), as the reader can check (see prob- 
lems), 





' A + BF + HC + HDF 


—H 


B + HD " 


J : 


F 


0 


/ 




-( C + DF ) 


I 


—D 



The implementation of the class of all stabilizing controllers in conjunction with 
a state estimate feedback nominal controller is shown in Figure 5.4. The input to 
the Q block is the estimation residual r , and the output s is summed with the state 
estimate feedback signal to give the final control signal. Clearly in this case, there 
is no additional computational requirement except for the implementation of the 
Q filter. 
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controller 

Closed Loop Properties 

Let us consider again the plant model P of (2.1) depicted in Figure 2.1 with 

P 2 2 = G. (5.15) 

We will next look at the closed-loop (matrix) transfer function of the plant model 
with K (Q) as the feedback controller, implemented as the pair (J, Q ) depicted 
in Figure 5.5. This is a generalization of the arrangement of Figure 5.1. The (ma- 
trix) transfer function of interest is that from the disturbance w to the disturbance 
response e. Referring to Figure 5.5, we will first derive the (matrix) transfer func- 
tion T, which consists of the plant P block (with P22 = G) and the J block. The 





FIGURE 5.5. Closed-loop transfer functions for the class of all stabilizing controllers 
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various equations are listed as follows, see also (5.10), (5.15). 



M 




>11 


a 2 ] 


w 




P21 


P 22 J 


M 




pit 


/t 2 ir 


W 




\_J21 


j22 \ I 



s=Qr. 



Pi 1 

p 2l 

K 

y- 1 




(5.16) 



We proceed by eliminating the variables y and u using G = M~ X N and K — 
V~ l U. We have, after a number of manipulations involving the double Bezout 
equation (4.12), noting a key intermediate result 



(/ - JuP^y 1 = (/ - V~ l UNM~ l r l = MV, 



that 



yi |v 


rj, Vi! 


Tn _ 


\Pn + PnUMP 2 i 


PnM~ 


= T 


; T = 

Tn 


T 2 2_ 


L mp 2 i 


0 



(5.17) 

It is interesting to note that T22 = 0 regardless of the plant or the nominal con- 
troller used. A direct result of this property is that the closed-loop (matrix) transfer 
function from w to e is given by 



e = F q w; Fq = T n + T 12 QT 2 i, (5.18) 

which is affine in the (matrix) transfer function Q. Of course, with T, Q e RFIoo, 
then Fq € RH 0 Q . Actually, because of the linearity in Q of the disturbance re- 
sponse (matrix) transfer function Fq, it is possible to construct convenient op- 
timization procedures to select stable Q to minimize the disturbance response 
according to reasonable measures. In this way, optimum disturbance rejection 
controllers are achieved with the search restricted to the class of all stabilizing 
controllers. This affine nature is also useful for numerical optimization, see Boyd 
and Barratt (1991). An adaptive version of this observation is presented in Chap- 
ter 6. 

It is interesting and useful for work in later chapters to examine the state space 
description for T in the case 
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Bi 


B 2 ' 


P : 


Ci 


Du 


Dn 




. C2 


D21 


D 22 . 



with state estimate feedback controller (4.17). Again we work with P22 = G, 
so that connecting with earlier notation G = NM~ l — M~ l N and B2 — B, 
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C 2 = C, simple manipulations give, under (4.18), (4.19) 
A + B 2 F 



’ A + HC 2 


B\ + HD 2 1 


C 2 


£>21 



Further manipulations which the reader can check (see problems) give an expres- 
sion for T\ 1 = P] 1 + P 12 UMP 2 1 , and since T 22 = 0, there follows 



A + B 2 F 


-hc 2 


-HD 21 


b 2 


0 


a + hc 2 


B\ + H D 2 \ 


0 


Ci + D l2 F 


Cl 


D n 


£>12 


0 


c 2 


£>21 


0 



An interesting special case is when the disturbance and disturbance response 
enter as in Figure 3.2, so that 



Then simple manipulations give 



[VA UN J 
\n m] 



We now consider other input/output relationships and stability properties. In 
the first instance, let us look directly at the possible input/output (matrix) transfer 
functions for the scheme of Figure 5.2, denoted (G, J, Q ). Simple manipulations 
give internal stability conditions as 



u T’f [;][- 
fc] [s *] [; "1 



Equivalently, Q e RH 0 0 . This result tells us that K(Q), when stabilizing for G, 
can be implemented as the ( J , Q ) control loop without loss of stability. 

It is straightforward to generalize the above stability results, including the re- 
sults of Theorems 3.1 and 5.1 to cope with the nested controller arrangements of 
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Figure 5.5, denoted ( P , J, Q). Necessary and sufficient conditions for stability 
are that the pairs 



/ To 0 1\ /To ol\ 

( P, IT, are stabilizing. (5.23) 

v L° K ^\) \ L° q \) 



All Stabilizing Feedforward/Feedback Controllers 

We have earlier introduced the feedforward/feedback controller as a feedback 
controller for an augmented plant. We will show in this section that applying the 
results of the above subsection in this case allows us to generate the class of all 
stabilizing feedforward/feedback controllers. 

Let us recall the augmented plant model of (3.9) and the corresponding nominal 
feedforward/feedback controller as 

= [“]• = [*/ 4 ( 5 - 24 ) 

Consider coprime factorizations for the plant G as in (4.1) and nominal feedback 
controller K as in (4.2) such that the double Bezout identity (4.12) is satisfied. 
Then coprime factorizations for the augmented plant and nominal controller 
are given by 



= ~“ 1 ~= '. =~“ 1 ~= (5.25) 

with rational proper stable factorizations: 

-[;} ■[:]. 

-.p "l. 

|_0 M\ 

= t/], ~ =\_Uf c] , * 

= ['. °l - = 4 

L NUf V \ 

U f = VK f . 



It is readily checked that these factorizations satisfy the corresponding double 
Bezout equation 



. II ]■[ IS 414 



(5.27) 
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FIGURE 5.6. A stabilizing feedforward/feedback controller 



Note that a coprime factorization for Kf is Kf = V 1 Uf so that any unstable 
modes of Kf are necessarily that of unstable modes of K. The relevant block 
diagram is depicted in Figure 5.6 where the feedforward block Uf is seen to be a 
stable system. 

The class of all stabilizing feedforward/feedback controllers parameterized in 
terms of is then given by 

( ) = [Kf( ) K( )]> V( r 1 [t//( ) U( )]. (5.28) 

The arrangement is depicted in Figure 5.7(a) with an augmented given by 

K f K V~ l 

—V~ 1 NU f V~ l -V-'N 

This arrangement can be reorganized to the scheme depicted of Figure 5.7(b) 
where J is that of (5.10) for the feedback case. The class of all stabilizing feedfor- 
ward/feedback controllers then consists of the corresponding feedback controllers 
and another stable block Qf in parallel with the nominal stable feedforward filter, 
also termed a precompensator, U f. 



(5.29) 




(a) 



(b) 



FIGURE 5.7. Class of all stabilizing feedforward/feedback controllers 











2.6 All Stabilizing Regulators 
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A subset of the class of all stabilizing controllers is those that can regulate a distur- 
bance response to a particular class of disturbances to zero asymptotically. Distur- 
bance classes of interest could be, for example, the class of constant disturbances, 
the class of sinusoidal disturbances of known period, or the class of periodic dis- 
turbances of known period with harmonics up to a known order. Of course, the 
class of stabilizing regulators for certain disturbance classes and responses may 
be an empty set. Let us proceed assuming that this class is nonempty. 

To conveniently characterize the class of all stabilizing regulators for a class of 
disturbances, it makes sense to exploit the knowledge we have gained so far as we 
illustrate now by way of example. 

Consider the class of constant disturbances. In this case, for models as in Fig- 
ure 2.1, the disturbance Wk is a constant of unknown value, and our desire is that 
the disturbance response e approach zero asymptotically for all w in this class. 
In order to proceed, let us modify the first step in our objective to requiring that 
the summed response eu = e < be bounded (in £2) for bounded input distur- 
bances w (in I2). 

Now this BIBO stability for linear systems is equivalent to requiring that con- 
stant inputs w give rise to asymptotically constant responses e, and in turn asymp- 
totically zero responses e, as desired. 

We conclude that the class of all regulators, regulating a disturbance response 
ek to zero asymptotically for constant input disturbances is the class of all sta- 
bilizing controllers for the system augmented with a model of the deterministic 
disturbance itself, namely a summing subsystem, such that the augmented system 
disturbance response is = Y^=i e i- 

It is not difficult to check that the class of stabilizing regulators includes an 
internal model of the constant disturbances, namely a summing subsystem. 

Sinusoidal disturbances and periodic disturbances can be handled in the same 
way, save that the augmentation and consequent internal model now consists of a 
discrete-time oscillator at the disturbance frequency, which is of course a model 
of the deterministic disturbance. 

More generally, any deterministic disturbance derived from the initial condition 
response of linear system model, can be appended to the disturbance response e in 
the design approach described above to achieve regulation of such disturbances. 
The consequence is that in achieving a stabilizing controller design of the aug- 
mented system, there will be an internal model of the disturbances in the con- 
troller. For further details for characterizing the class of all stabilizing regulators, 
see Moore and Tomizuka (1989). 

Of course, if regulation of only a component of the disturbance response e is 
required, then the augmentations of the deterministic disturbance model need be 
only to this component. This observation then leads us back again to the selec- 
tion of the appropriate component of e so that regulation can be achieved. This 
component is frequently the plant output or a tracking error or filtered versions of 
these since, in general, it is not possible to have both the control u and the plant 
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output y (or tracking error) regulated to zero in the presence of deterministic dis- 
turbances. 



Main Points of Section 

In this section, we examine the use of fractional representations of transfer func- 
tions to parameterize the class of all stabilizing controllers for a plant in terms of 
any stabilizing controller for the plant and a stable filter Q. The (matrix) trans- 
fer function of the closed-loop system is affine in the stable filter Q. The class 
of stabilizing controllers has convenient realizations in the feedforward/feedback 
(two-degrees-of-freedom) situation as well as in just the feedback (one-degree- 
of-freedom) situation. In particular, an interesting special case is where the filter 
Q augments a stabilizing state estimate feedback controller. 



2.7 Notes and References 

In this chapter, we have set the stage for the rest of the book by defining the plant 
representations with which we are working. We have introduced the concept of a 
nominal stabilizing feedback controller, and feedforward/feedback controller, for 
such plant models. The entire class of stabilizing controllers for the plant mod- 
els is then parameterized in terms of a stable filter Q using a stable, coprime 
fractional representation approach. It turns out that this approach gives rise to 
closed-loop transfer functions that are affine in the stable filter Q. This result 
underpins the controller design methods to be explained in the rest of the book. 
In effect, our focus is on techniques that search within this class of stabilizing 
controllers for a controller that meets the performance objectives defined for the 
controller. 

More information on definitions and manipulations of the plant model can 
be found in Ogata (1987), Astrom and Wittenmark (1984), Franklin and Pow- 
ell (1980), Boyd and Barratt (1991) and Doyle et al. (1992). A detailed exposition 
on linear system theory can be found in Kailath (1980) and Chen (1984). 

Algebraic aspects of linear control system stability can be found in Kucera 
(1979), Nett (1986), Francis (1987), Vidyasagar (1985), McFarlane and Glover 
(1989) and Doyle et al. (1992). For the factorization approach to controller syn- 
thesis, the readers are referred to Youla, Bongiomo and Jabr (1976b), Kucera 
(1979), Nett, Jacobson and Balas (1984), Tay and Moore (1991), Francis (1987), 
and Vidyasagar (1985). For a general optimization approach exploiting this frame- 
work we refer the reader to Boyd and Barratt (1991). 

Some of the third author’s early works (with colleagues and students) in the 
topic of this chapter are briefly mentioned. Coprime factorizations based on Lu- 
enberger observers which could be of interest when seeking low order controllers 
are given in Telford and Moore (1989). Versions involving frequency shaped (dy- 
namic) feedback “gains” F and FI are studied in Moore, Glover and Telford 
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(1990). The difficulty of coping with decentralized controller structure constraints 
is studied in Moore and Xia (1989). Issues of simultaneous stabilization of mul- 
tiple plants are studied in Obinata and Moore (1988). 

Generalizations of the work of this chapter to generate the class of all stabiliz- 
ing two-degree-of-freedom controllers are readily derived from the one-degree-of 
freedom controller class theory, see Vidyasagar (1985), Tay and Moore (1990) 
and Hara and Sugie (1988). Two-degree-of-freedom controllers incorporate both 
a feedforward control from an external input as well as feedback control as 
in this chapter. The most general form has a control block with output driv- 
ing the plant and two sets of inputs, with dynamics coupling these. The con- 
trollers are parameterized in terms of a feedforward Q\ e RH 0 Q and the feed- 
back <22 e RH 0 o. Indeed = [Gi Qi] e RH 0 0 can be viewed as the pa- 
rameterization for the class of all stabilizing one-degree-of-freedom controllers 
for an augmented plant [oc 1 ]'. An application and more details are given in 
Chapter 8. 

Generalizations of the work of this chapter to the class of all model matching 
controllers is studied in Moore, Xia and Glover (1986). For this it is required that 
the closed-loop system match a model. Further details are not given here save that 
again there is a parameterization Q e RH 0 G . 

Other generalizations to the class of all stabilizing regulators can be achieved, 
see Moore and Tomizuka (1989). These regulators suppress asymptotically ex- 
ternal deterministic disturbances such as step function inputs and sinusoidal dis- 
turbances. These can arise as discussed in the next chapter. Again the parame- 
terization is a <2 e RHoq. Here however, the regulators/controllers must have an 
internal model of the disturbance source. 

Then too there are generalizations of all these various stabilizing controller 
classes for time-varying systems, being still linear, see for example Moore and 
Tay (1989a). In fact, the theory of this chapter goes through line by line replacing 
transfer functions by linear system operators and working with BIBO stability, 
or equivalently now exponential asymptotic stability. Indeed, our [ ] notation 

really refers to a linear system with matrices A, B,C, D which as it turns out 
can be time varying. Such results are used in Chapter 8, where also nonlinear 
generalizations are discussed. 



Problems 

1. Derivation of plant model. 

Consider the plant model of Figure 2.1 with P22 = G and P21 = / . Con- 
struct the blocks Pi 2 and P\\ in terms of G so that e = [ ? u ] where A. is a 
constant. 

Notice that with e interpreted as a disturbance response and G restricted to 
a linear plant model, a control objective that minimizes the 2-norm of e will 
lead to the linear quadratic design problem. 
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2. Transfer function to state-space transformation. 

Consider a plant G with polynomial description given as follows. 

Mz~ l )yic = B{z~ l )u k , 

A(z~ l ) = 1 + a lZ ~ l + a 2 z~ 2 , B(z~ l ) = b x z~ l + b 2 z~ 2 . 

By considering a related system given by A(z _1 )f* = u k - i, show that 
y k = B(z~ 1 )u k . Show further that the plant G has a state-space description 
given by 



x k+i = ^ 2 j x k + u k , y k = [bi Z> 2 ] x k . 

This particular state-space realization for the transfer function G is some- 
times referred to as the controller canonical realization, although there is 
no universal agreement to the name. Note that this is just one of the many 
possible state-space realizations for the the transfer function G. The inter- 
ested readers are referred to texts on linear systems such as Kailath (1980) 
and Chen (1984) for a detailed exposition. 

3. Elimination of uncontrollable and unobservable modes. 

Consider the systems W\ and W 2 of (2.9) with A\ = A 2 , B\ = B 2 and 
their sum in (2.10). Show that after elimination of uncontrollable and un- 
observable modes, the sum of the systems can be represented as 



Ai 


Bi 


Ci + C 2 


D\ + D 2 _ 



With W ] , as in (2.9) with D invertible, verify that W\W t 1 = W ] 1 W\ has 
a minimal representation [ q / ]• Discuss internal stability. 

4. Internal stability. 

In the series connection of two scalar systems W \ , W 2 , with minimal repre- 
sentation given by (2.9), show that the series connection W\ W 2 is minimal 
if and only if the transfer functions of W\ and W 2 do not have any zeros 
common with poles of the transfer function of W 2 and W\, respectively. 
Show that the single-input, single-output plant-controller pair (G, K) given 
by 



0.4z -1 - 0.44z“ 2 
“ 1 -2z~ l +0.96z- 2 ’ 



K = 



-5 + 2.4 z” 1 
1 - l.lz" 1 ’ 



is not internally stable in a feedback loop. Explain your findings in term 
of classical pole/zero cancellations. Does the following multi-input, multi- 
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output plant and controller form a stabilizing pair in a feedback loop? 




Are there any pole/zero cancellations between the plant and controller? 
How are transmission zeros of a multi-input, multioutput transfer function 
defined? 

5. Implementation of feedforward/feedback controllers. 

A two degrees-of-freedom controller given as 

-[*/ ^ = [ ,,-,,^-0,,- ' , „!£V-££., ] 

is implemented as in Figure 7.1. Explain why the structure used in the figure 
is not suitable for implementing K. Suggest a more suitable structure. 




FIGURE 7.1. Signal model for Problem 5 



6. Coprime factorizations. 

Verify that the representations (4.18) and (4.19) indeed are factorizations 
for G and K. Establish that in the case that the plant of (4.16) has no direct 
feedthrough, that is D = 0, then we can select a gain matrix L where AL 
is a stabilizing output injection such that the feedback controller K is now 
given by 



’ A + BF + ALC + BFLC 


-( AL + BFL ) " 


F + FLC 


-FL 



Note that this controller has a direct feedthrough term — ( FL ), although the 
overall control loop transfer function GK is still strictly proper. 

Also, for the plant G with the controller (7.2), stable coprime factorizations 
are given by 




' A + BF 


B 


—{A + BF)L ' 


F 


I 


-FL 


C 


0 


I 
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and 




' A + ALC 


-B AL ' 


F + FLC 


I FL 


C 


0 1 



(7.3) 



7. Coprime factorizations. 

For the multi-input, multioutput plant and controller given in Problem 4, 
suggest stable, coprime factorizations in |z| < 1. 

8. Coprime factorizations. 

Verify that the coprime factorizations given in (4.22) and (4.23) satisfy the 
double Bezout equation. 

9. Implementation of the class of all stabilizing controllers. 

Starting from the expression for J given in (5.10), verify that the structure 
of Figure 5.3 implements the class of all stabilizing controllers K(Q ) for 
the plant G, given in (5.9). Verify the state space representation of J given 
in (5.14) and T given in (5.22). Hints: In deriving (5.14), recall that K is 
given from (4.17), and note that from (4.18), (4.19) 



’ A + BF 


-H 






C + DF 


I 






’ A + BF 


B 






C + DF 


D 






’ A + HC 


-( B + HD ) ' 


F 


/ 



Apply the manipulations of (2.11)-(2.21) to obtain (5.14) from (5.10). In 
verifying (5.22), check that indeed T 22 = 0, and that Tn and 721 are given 
by (5.20), (5.21), and that T n = P u + P 12 UMP 21 = Pn + PnUT 2 \ is 
given by 





A + B 2 F 


-HC 


-HD 21 


Tn = 


0 


A + HC 


Bi + HD 21 




_ Ci + DnF 


Ci 


Du 
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Key intermediate steps in the derivation are spelled out below. 





A 


b 2 f 


0 


" A + HC 


hc 2 


HD 2 \ 


T U = 


0 


a + b 2 f 


-H 


0 


A 


Bi 




. Ci 


d 12 f 


0 


. c 2 


c 2 f 


D 21 



A 


Bi 


. C i 


Du _ 



A 


0 


0 


0 


0 


Bi 


0 


A 


b 2 f 


0 


0 


0 


0 


0 


a + b 2 f 


-hc 2 


-hc 2 


-HD 2 1 


0 


0 


0 


a + hc 2 


hc 2 


HD n 


0 


0 


0 


0 


A 


B\ 


. Cl 


Cl 


D V _F 


0 


0 


Du 



Now add block row 5 to block row 4 and subtract block column 4 from 
block column 5, then delete block row 5, and block column 5 (representing 
unobservable modes). Next subtract block column 2 from block column 1 
and add block row 1 to block row 2, then delete the block first row and 
column (again representing unobservable modes). Next add block column 
1 to block column 2 and subtract block row 2 from block row 1, then add 
block column 1 to block column 3 and subtract block row 3 from block row 
1, and finally delete block row 1 and column 1 (representing uncontrollable 
modes) to give the required form for T\ \ . 




CHAPTER 3 



Design Environment 



3.1 Introduction 

In this chapter, we focus on the environment in which our controller designs are 
carried out. We begin with a discussion of the various types of disturbances in 
the design environment. We introduce models for disturbances, both predictable 
and nonpredictable, and the concept of norms to measure the size of disturbances. 
We then go on to discuss plant uncertainties in the modeling process. General 
frequency domain and time domain model uncertainties are discussed. 

Next, we introduce a special form of frequency-shaped uncertainty motivated 
by the characterization of the class of all stabilizing controllers. We examine the 
class of all plants stabilizable by a nominal controller. This is dual to the case 
of the class of all stabilizing controllers for a plant, and is characterized in terms 
of a (matrix) transfer function, denoted S. It turns out that there is an interesting 
relationship between an actual plant in the class of plants stabilized by a nominal 
controller, the nominal plant description and the parameter S. The parameter S is 
a frequency-shaped deviation of the actual plant from the nominal plant with the 
frequency- shaping emphasizing the operating frequencies in the nominal closed 
loop and the actual closed-loop systems. 



3.2 Signals and Disturbances 

In this section, we first suggest models for some commonly encountered pre- 
dictable or deterministic disturbances. This is followed by a discussion on nonde- 
terministic or stochastic disturbances. The concept of norms to measure the size 
of such noise is introduced. Much of this material may be familiar to readers, but 
is included for completeness and reference. 
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Deterministic Disturbance Model 

A deterministic disturbance is one where its future can be perfectly predicted 
based on past observations. A number of useful disturbance models can be con- 
structed and the most appropriate one depends on the nature of the disturbance 
environment and control design objectives. Here we discuss a few of the more 
commonly encountered disturbances in applications and provide a general for- 
mulation for these. The disturbances of interest can be modeled by homogeneous 
(time-invariant) state space or input/output models. In turn these models can com- 
bine with the plant models to give composite models of the plants in conjunction 
with the disturbance environment. The inclusion of disturbance models is essen- 
tial in most control applications. The ubiquity of integral action in control arises 
from the need to have accurate set point regulation, or equivalently, the rejection 
of a constant disturbance. 

Deterministic disturbances characterized by a finite, discrete frequency spec- 
trum can easily be modeled using autonomous state space models. Typical exam- 
ples follow. 

DC offset 

A constant disturbance can be modeled as: 

(1 — q~ l )dk = 0, do = constant, k— 1,2,... (2.1) 

where q~ x is the unit delay operator (i.e. q~ l dk = dk- 1). 

Drift at Constant Rate 

A ramp like disturbance has a discrete-time model given by 

(1 — 2q _1 + q~ 2 )dk = 0, do = P, d-i = P - aT , k= 1,2,... 

( 2 . 2 ) 



where T is the sampling interval. Here dk = otTk + fi. 

Sinusoidal Disturbance 

A single frequency sinusoidal disturbance can be modeled as the solution of the 
difference operation. 

(1 — (2 cos coT)q~ l + q~ 2 )dk = 0, do = a, d-i=acoscoT. (2.3) 

The solution of (2.3), namely dk = a cos (coTk), corresponds to sampling a 
continuous-time disturbance d{t) — a cos {cot) with sampling period T . 

More General Disturbances 

Any finite combination of sinusoidal, exponential or polynomial functions of time 
can be represented as the solution of an appropriate difference equation. Such 
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disturbances may be modeled as: 

r (q~ l )d k = 0, (2.4) 

suitably initialized where 

r( z - l )=l + y lZ - l + ... + y n z- n . (2.5) 

The zeros of the polynomial F determine the discrete frequency spectrum con- 
tained in the disturbance. Alternatively, a state space model can be used: 

—yi 1 0 ... 0 

: 0 : 

: : •• 0 

: 0 ... 0 1 

-y„ 0 0 

d k = [ 1 0 ... o]jc*. 

The model for a periodic disturbance with known period but unknown harmonic 
content is in general infinite dimensional. In practice, such a model is approxi- 
mated by a finite-dimensional model as above. 

Stochastic Disturbance Models 

In a control environment, deterministic disturbances can be dealt with adequately 
by appropriate control action. Many disturbances however are not predictable. 
A controller solely determined on the basis of models without allowing for any 
uncertainty could fail to achieve acceptable performance in an environment that 
differs from its design environment. In order to determine a practical controller, 
it is helpful to include unpredictable disturbances in the signal environment. Sto- 
chastic disturbances can be modeled using stochastic processes. In the context of 
linear system design, simple stochastic processes whose statistical characteriza- 
tion can be expressed in terms of first and second order means suffice. Most of the 
design methods in linear system theory, linear quadratic methods, l \ optimization 
and Hoc can all be motivated and obtained in a purely deterministic setting as 
well as in a stochastic setting. This is because first and second order means can be 
defined via time averages rather than expected values over some underlying prob- 
ability space defining the stochastic variable. This has been explored in depth in 
Ljung and Soderstrom (1983). In our setting, it suffices to consider disturbances 
like u>k,k = 1,2,... that can be characterized in the following way: 

Definition. Bounded, zero mean, white noise process Wk- For some constants b, 
b\,b2>0 and 0 < y < 1 and all sufficiently large N: 
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1- \m\<b, 

(Bounded). 

2 | N | 

|I>*| < Mr y , (2.7) 

(Zero mean). 

3 | ^ | 

Y w k w' k _ m -WS m \<b 2 N y , m = 1,2,... 

£=m+l 

(Uncorrelated and with variance W). 

Here 5* = 1 if k = 0 and S k = 0 for k / 0. 

The frequency spectrum of this noise is flat, hence the term white noise. 

Definition. The bounded sequences w k , v k , k= 1, 2, ... are said to be uncorre- 
lated if for all sufficiently large N, 

I N I 

Y w kV k _ m < bsN v + b4, m — 1 , 2 ,... 



for some positive constants b^, > 0 and 0 < y < 1. 

If it is important to limit the frequency content of the noise, a filtered or col- 
ored noise sequence can be derived from w k via an appropriate linear filter that 
accentuates the frequency spectrum of interest as follows: 



x k +\ = A f x k + B f w k , 
y k =C f x k + D f w k . 



( 2 . 8 ) 



If w k in (2.8) satisfies the white noise condition (2.7) and Ay is a stable matrix, 
then y k is bounded, has zero mean, but is correlated over time. Also, y k has a well 
defined autocorrelation. Its frequency spectrum is given by: 



*y,m = (Cf(le- je -A f ) l B f + D f )(Cf(lei e -A f ) X B f + D f ), 



where 9 e [0, 27r). 



Norms as Performance Measures 

The performance of a controller can only be expressed via characteristics of the 
signals arising from the controlled loop. An important characteristic of a signal is 
its size. Norm functions are one way to determine the size of a signal. 
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Consider sequences d := {dk,k — 1,2,...} mapping to ".This constitutes 
a linear space of signals £( , "), where addition of signals and multiplication 
with a scalar are defined in the obvious way: d + e = {dk + ek,k = 1,2,...}; 
ad m {adk, k = 1,2,...} ford, e e l{ , ") and a e .A norm ||-|| for signals 
has to satisfy the following properties. It maps from the signal space to the positive 
reals ||<i|| > 0. It measures zero only for the zero signal, that is \\d\\ = Oif and only 

if d = 0 or dk = 0 for all k = 1,2, It scales appropriately \\ad\\ — a ||<i||, 

a € and satisfies the triangle inequality || d + e || < ||<i|| + ||e||. Typical examples 
are the l p norms. The l p norm is defined as follows: 

/ JL " \Vp 

W\ P = • d k = (d hk ...d nA ) - p > 0. 

Commonly used t p norms are the norm, which measures the maximum value 
attained by the sequence; the l\ norm which measures the sum of absolute values 
and the I2 norm which is often referred to as an energy measure: 

Halloo = sup max \di,k \ , 

( 2.9) 

k=\ ;=l v ’ 

/ N n \ 1/2 

In discrete time, signals with finite l\ or £2 norm converge to zero as time pro- 
gresses; such signals are of a transient nature. 

A less common norm is: 

||d|| = sup max (\di k+\ — dj 1 ) + sup max \dik\- (2.10) 

k e 1=1,2 B V ’ ’ '/ ke i=l,2,...,»i ’ 

It measures not only the magnitude of the signal but also the rate of change of the 
signal. A signal bounded in this norm is not only of finite magnitude but also rate 
limited. 

Sometimes norms are not informative enough to measure the quality of a signal, 
for example any white noise signal with Gaussian distribution has infinite £ 0 0 and 
I2 norms. A more appropriate measure for such signals would be: 

/ 1 " 

ll^llrms = w l™ o ^E<4j • (2.11) 

This is not a norm however. Indeed any signal of finite duration, that is dk = 0 
for all k > k{) > 0, or more generally that converges to zero has zero root mean 
square (rms) measure. The rms value measures the average power content of a 
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signal. It is an excellent indicator for differentiating between persistent signals 
and nonpersistent ones. 

For vector valued signals the different components in the vector may have dif- 
ferent significance. This can be made explicit in measuring the size of the signal 
by weighting the different components in the signal differently, for example 

\\d\\ = \\Pd\\ 0O , (2.12) 

is a norm for any symmetric positive definite matrix P of appropriate dimension. 
In particular, P — diag ( p\ , p 2 , . . . , p n ) with pi > 0, i = 1 . . . n allows for 
individual scaling of each component of the signal d. Proper scaling of the signals 
is extremely important in practical application. 

In concluding this section, we remark that similar norms can be defined for 
continuous-time signals in very much the same fashion. We refer the reader to 
Boyd and Barratt (1991) or Sontag (1990). 



Main Points of Section 

In the context of discrete-time signals, we see that the concept of a norm is 
useful measure of the size of a signal. Commonly used norms are introduced. 



3.3 Plant Uncertainties 

Models are only approximate representations for physical plants. Approximations 
are due to a number of simplifications made in the modeling process, for example 
we use linear, time-invariant models to represent systems which are really time 
varying and nonlinear. As an illustration, consider a simple linear, time-invariant 
model for the yaw control of an airplane. Complete description of the airplane’s 
dynamics are nonlinear in that they depend on speed, altitude, and mass distri- 
bution; factors that vary over the flight envelope of the airplane. A simple linear 
time-invariant model can at best capture only the essential behavior in the neigh- 
borhood of the one flight condition. 

Model-plant mismatch can be represented or characterized in a number of dif- 
ferent ways. The mismatch measure must capture the different behaviors of the 
model and the plant for a range of signals that can be applied to the model and 
the plant. This model-plant mismatch, also termed uncertainty, is therefore best 
measured via signals. Both time domain and frequency domain characterizations 
are important and play a role in control design. 

Model-plant mismatch is also an important factor in determining controller 
properties. A controller designed for the model must at least retain stable be- 
havior when applied to the plant, that is robustness to plant model uncertainty. 
In expressing robustness it becomes important to quantify the neighborhood of 
plants near the model that can be stabilized/controlled by the particular controller 
of interest. In this context, norms for models/transfer functions become important. 
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In this section we discuss concisely different model-plant mismatch character- 
izations and introduce some frequently used norms for models/transfer functions. 



Norms for Models 

When discussing differences between a plant and a model, it is important to be 
able to quantify this difference. A norm function is a suitable way of achieving 
this, at least for stable plants. Consider the class of stable rational transfer function 
matrices, with a realization: 



G : 



A B 



C D 



e RHoo. 



(3.1) 



Induced i p Norms 

One way of defining a norm for a plant model G is via the signals it links. Suppose 
that Gu is the output produced by applying the signal u to the model G, setting 
all initial conditions to zero as follows: 



Xk+ 1 = Axk + Bug *0 = 0, 


(3.2) 


(Gu)k = Cxk + Duk. 


(3.3) 


The l p gain of G is defined by 




IIG|| p _£„ := sup \\Gu\\ p . 


(3.4) 



It measures the gain between the input to the plant G and the output produced by 
the plant acting on its input. 

The too gain of the system G is equal to the l \ norm of the response of G to 
an impulse input uk = 8^ where 8k = 1 for k — 1 and 8k ^ 0 otherwise. This so- 
called impulse response is denoted g or g i , g2, • ■ ■ with go = 0, gi = CB + D, 
g2 = CAB, gj — CA l ~ 1 B for i = 2, 3, Indeed, we have: 

l|G|loo_ SIl = f>;|loo := kill- 

;=i 

Here ||g, is the induced oo-norm for the matrix gi . (See Appendix A). 

The £2 gain of the system G is also referred to as the norm or as the rms 
gain. 

l|G|| 2 - g „ = IIG|lrm S = SUp || Gm 11^ . 

l|H|lrms=l 

Observe that unlike for signals, || GUn^ is a true norm for the system G. It can be 
computed as: 

l|G|| 2 _£„ = sup cr max (g (e J0 )) , 

0<9<2jr v v '' 
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where er m ax(A) stands for the maximum singular value of the matrix A. In reluc- 
tant compliance with the literature, we at times denote || G II2— = l|G||oo? ar| d 
refer to it as the H 0 Q norm of the system G, see also Appendix B. 

The frequency domain interpretation of the £ 2 gain (or H 0 0 norm ) for a system 
is very natural in many applications. Unfortunately it puts the whole frequency 
spectrum on an equal footing. Often the gains in specific frequency ranges are 
more important than outside these ranges. A frequency weighted £2 gain can then 
be used: 



\\G\\ W = \\W 0 GWi\\ 2 _ gn 

= sup a max (w 0 GWi (eJ 0 )) . 
0<9<2n V \ // 

Here W„ and W; are frequency weighting systems; 



W 0 : 




Wj : 




respectively, at the output and input of the plant G. Without loss of generality, the 
realizations are symmetric: A a = A' 0 , D a = D' a , A; = A', and D, = D(. Also 
W a (e j0 ) > 0, Wj (e j0 ) > 0 for all 0 < 6 < 2n. The main motivation for having 
Wo, Wi with symmetric realizations is to interpret the frequency weighted norm 
as a proper norm, see Boyd and Barratt (1991). 



Computing the I2 Gain 

The f 2 gain can be approximated using state space methods. For the system G, 
we have that for stable G with a realization as given in (3.1) there exists an upper 
bound y such that 

II Gil 2 = Q< s e u P/ m ax (G (reJ 0 )) < y, 



if and only if there exists a symmetric positive definite P and matrices L, W 
satisfying 




0 Ol |~A B~ 
7 0 CD 

0 / L W 




This can be used as the basis for a numerical procedure to compute the £2 gain 
(Hoq norm) of a stable rational transfer function matrix. Starting with a suitably 
high value of y for which a positive semi-definite solution P of the above linear 
equation exists, then y is progressively reduced until semi-definiteness of the so- 
lution P fails. This procedure yields a least upper bound y, or equivalently, the £2 
gain, see Green and Limebeer (1994). 
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H2 norm of a system 

The £2 norm of the response to an impulse input is also often used as a measure 
of a system; this is the so-called H2 norm. 

\\G\\ 2 =\\G8\\ 2 =^ j 2 J G {e~ je ) G (e^ e ) d 0. (3.5) 

Here as before, 8k = 1 for k = 0 and % = 0 for k / 0. 

Parseval’s Theorem which equates energy in the time and frequency domains, 
is used here to establish the second equality in (3.5). It allows us to interpret the H2 
norm as the energy content of the output of the system subject to white noise — 
recall that white noise has a unit frequency spectrum. This simple equivalence 
allows one to interpret many design optimization schemes in an either purely 
time-domain deterministic, or frequency-domain stochastic context. 

As a final interpretation of the H2 norm, it is possible to show that 

II (7 II 2 - sup IIGu;^. 

' il« l2.=l 

This establishes a link between the l 0 0 norm of the time response to an I2 signal 
and a frequency defined 2-norm. 



Frequency Domain Uncertainty 

Expressing uncertainty in the frequency domain is appealing as it provides phys- 
ical insight. The £2 gain, especially the weighted £2 gain, is the preferred tool for 
this. 

We illustrate the ideas using stable single-input, single-output (SISO) stable 
plants. Consider: 



G : 



A B 
C D 



€ RHoo. 



The £2 gain (H 0 0 norm) of G is then simply the maximal amplitude in the Bode 
plot of G’s transfer function: 

||G|| 2 _ g „ = o max | c(e je l-A^ ' Z? + d| . 

When G is a model for a real plant G, the uncertainty associated with G can be 
expressed in terms of a weighting system W as: 

Aw = )G:||(G-G)W|| 2 _^<l}. 

In frequency terms the above inequality corresponds to: 

| G (e je ^ - G ( e ie ) | • | W (e je ^ | < 1 . 
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It simply states that the plant’s transfer function response at any one frequency can 
be found in a ball centered at the model’s response and with radius | W (e J0 ) \ . A 

large weight indicates small uncertainty, while a small weight indicates large un- 
certainty. This uncertainty description allows for both amplitude and phase errors 
in the transfer functions. 



Time Domain Uncertainty 

Besides having a nonexact model for the system’s transfer function from con- 
trol inputs to controlled variables, the response may be perturbed by other signals 
from other systems affecting the output variables. Such uncertainties are best cap- 
tured by a combination of signal norms and system norms. Our standard two port 
model for the plant allows for such uncertainties, see Figures 2.2.1 and 2.2.2*. 
With reference to Figure 2.2.2, w\,W2 can be interpreted as disturbances, G as an 
approximation for the true system linking control input u and controlled output 
y. Interpreting the uncertainty associated with G in £2 gain terms, it then makes 
sense to interpret the use of w\ and wi in terms of rms measures. The uncertainty 
environment associated with the control loop could then be expressed as: 

||(G-G)#J rms < 1, Mrms < W 2 , ||U>1 linns < % 

The effect of additive disturbances in a linear system is to deteriorate the control 
performance. Such signals can not effect stability. In the context of H 0 0 optimiza- 
tion, discussed in Chapter 4, we introduce the concept of a worst case disturbance, 
which results in a worst case performance. 

Main Points of Section 

Norms for linear systems such as the induced l p norm and Hi norm, and other 
measures such as rms measures, are useful for representing uncertainty in plant 
models. Frequency domain or time domain representations of this uncertainty are 
useful to achieve robust controller design. 



3.4 Plants Stabilized by a Controller 

In this section we apply the characterization of the class of all stabilizing con- 
trollers for a nominal plant to the dual situation of characterizing the class of all 
plants stabilizable by a given controller. In particular, we begin with the inter- 
nally stable closed-loop system of Figure 2.3.2 formed by the nominal plant G 
and a stabilizing controller K. We then classify the entire class of plants G(S) 
parameterized by a (matrix) transfer function, referred to as S e RH 0 c , that can 
be stabilized by the controller K . 

*In referencing figures from another chapter, the first of the three numbers indicates the chapter. 
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Class of All Plants Stabilizable by a Controller 

Let us consider the plant G e R p of (2.2.4) + and its corresponding stabilizing 
controller K. Let the stable, coprime factorizations for both G and K that sat- 
isfy the double Bezout identity of (2.4.12) be given by (2.4.1) and (2.4.2) of the 
previous chapter, and repeated here for convenience as 



G = NM~ l = M~ 

k = uv~ i = v~y 



N, M,N,M e RHoo 
U, V,U,V e RHoo 

-ol 



\\ -fir* u }J M "lr* °i. ( 4 . 2 ) 

\_-N M J \_N vj [N V ] \_~ N M \ L° 7 J 

The class of all proper plants stabilized by the controller K can then be character- 
ized as in the following theorem. 

Theorem 4 . 1 . For the factorizations of (4.1) the class of all proper linear plants 
stabilized by the controller K can be parameterized by an arbitrary S e RH^ as 



where 

G(S) = N(S) = N + VS, M(S) = M + US, or 

G(S) = M(S)~ 1 N(S ); N(S) = N + SV, M(S) = M + SU. 



Proof. The proof of the theorem follows closely the development of the class of 
all stabilizing controllers for a proper plant in the previous chapter. Interchanging 
the role of G and K , (2.5.7) of the previous chapter is written as 




From (4.4), it is clear that any plant parameterized by S e RH 0 0 is stabilized by 
the controller K. Conversely, from (4.4) and the double Bezout identity (4.2), the 
(matrix) transfer function S is given as 




If both G and G(S) are stabilized by the controller K, then the closed-loop (ma- 
trix) transfer functions of (G(S). K ) are stable. In this case, S as given in (4.5) 
satisfies S 6 RH a 0 □ 



fin referencing equations from another chapter, the first of the three numbers indicates the chapter. 
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FIGURE 4.1. Class of all proper plants stabilized by K 
As in (2.5.9), G(S ) can be re-expressed via the double Bezout identity of (4.2) 



G(S) = G + M~ l S(I + M~ 1 US)~ 1 M~ 1 . (4.6) 

which can then be reorganized as in Figure 4. 1 with 

From (4.6), note that G(S ) consists of the sum of G, the nominal plant and a term 
involving the parameter S £ RH 0 G . In the next subsection we actually interpret 
the parameter S as a frequency- shaped mismatch between the actual plant and the 
nominal plant model. 

Interpretation of S 
From (4.6), S can be written as 

S = M{G{S ) - G)M(I + M~ l US ) 

= M{G{S) - G)(M + US) (4.8) 

= M(G(S ) - G)M(S), 

or alternatively, as 

S = M(S)(G(S) - G)M. (4.9) 

We see that any S £ RH 0 0 will generate a unique G(S) that forms a stabiliz- 
ing pair (G(S), K), and conversely, for each stabilizing pair (Gi, K) there exists 
a unique S e RH 0 Q which generates G i = G (S). It is immediate from (4.8) 
and (4.9) that S can be interpreted as the difference in the (matrix) transfer func- 
tions between the actual plant G(S) and the plant G, frequency- shaped by either 
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M(S), M or M, M(S). It is important to realize that since the factorizations are 
not unique, the frequency- shaping is not unique. Let us explore the nature of the 
frequency shaping. 

From the Bezout identity, we have 

VM(S) - UN(S ) = /, 

V(I - KG(S))M(S ) = /, 

M(S ) = (/ - KG(S))~ l V~\ (4.10) 

Similarly 

M(S)=V 1 (/ — G(S)K) '. (4.11) 



It is immediate that M (S) or M (S) provides frequency shaping of (G(S) — G) to 
emphasize the actual operating frequencies in the closed loop. Of course, taking 
S = 0 in (4.10), (4.11), we see that M or M provides frequency shaping for 
(G(S) — G) to emphasize the operating frequencies in the nominal closed loop. 
Note that M and M(S), M and M(S) are determined by the plant G(S), the model 
G and the nominal controller K . 

Example. Let us now demonstrate some of the results above for the case where 
the underlying actual process has a scalar auto-regressive, moving average, exoge- 
nous input model (ARMAX model) description, in operator (transform) notation 



where 



(z x ) ^ = 3i *) 

(z -1 )’ W (z -1 )’ 

I ) = l+aiz 1 + • ■ ■ + dmZ m , 

J ) = bo + b\z~ l 1 1- bnZ~ n , 

l ) = \ + C lZ - 1 + - +CpZ~ P . 

Assume that the nominal plant G is available and is given by 




(z X ) = 1 +a\z 1 H \-a m z m , 

(z -1 ) = bo + biz~ l -\ h b n z~ n . 



G = 

~ (z _ 
"(z" 
"(z“ 



(4.12) 



(4.13) 



(4.14) 
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Let the factorizations for G and G be given by 
M = M 
N = N 
M(S ) = M(S) 

N (S) = N (S) 

where D(z _1 ) and Z)(z _1 ) are stable polynomials derived from the factorizations 
of (2.4.18), (2.4.19), (2.4.22) and (2.4.23). These reflect the nominal closed-loop 
poles and can be appropriately designed through the design of some nominal con- 
troller K . 

Now let us assume that S is parameterized by polynomials s(z -1 ), s(z _1 ) 

S = frac 5 (z _1 ) s(z _1 ). (4.16) 



Then from (4.8), we have 



s(z _1 )= "(z _1 ) <z-l). 

s(z _1 ) = "(z -1 ) (z-l)-'(z- 1 ) (z-1), 
M{S)G W = 

s(z“ 1 )= '(z“ x ) (z” 1 ). 



(4.17) 



When designing controllers for an actual plant where an a priori model G is 
known, there is advantage in specifying dynamic uncertainty in terms of S and 
thus working with a plant description G(S). Note that for an actual plant G — 
G(S), the order of S may be higher than the order of the plant G or its actual model 
G. However through proper frequency- shaping, or equivalently through a good 
initial robust design for a stabilizing controller on the nominal plant, we may well 
have an S that can be fairly accurately described by a low order approximation, or 
perhaps a low gain S where its complexity is not of any real consequence. These 
ideas are best illustrated by an example. 

Example. In this example we consider an eighth-order nominal plant G. The 
magnitude and phase plots are shown in Figure 4.2. This nominal plant has the 
characteristics of a band pass filter, more precisely an elliptic filter. The actual 
G(S ) is a perturbed version of G and the magnitude/phase plots are shown along 
side the nominal plant G. 

An LQG nominal controller K (see details in Chapter 4) is designed for the 
nominal plant. This nominal controller stabilizes both the nominal plant and the 
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Normalized frequency 




Normalized frequency 



FIGURE 4.2. Magnitude/phase plots for G, S, and G(S) 



actual plant G(S). Using the techniques described in this section, we compute the 
corresponding S. The magnitude/phase plot for S is also shown alongside the plot 
for G and G(S) in Figure 4.2. 

At first glance, it may appear that S is too complex to work with, being perhaps 
more complex than either G or G(S). From our derivation in this section, the 
complexity of S given any arbitrary G or G(S) would be twice that of either G 
or G(S). Intuitively, consider the adverse situation where the nominal plant G 
is a poor approximation to the actual plant. In this case, the S which together 
with G give the actual plant will have to first “cancel” the dynamics of G before 
“constructing” the dynamics of the actual plant. In such a case, we would expect 
the complexity of S to be the complexity of the nominal plant plus that of the 
actual plant, and with no possibility to even approximate S by a lower complexity 
object. 

However, in the event that the nominal plant is a good approximation of the 
actual plant in a certain frequency band, the situation could be rather less daunting 
since some of the complexity in S could be of negligible significance since its 
magnitude would be relatively small. Take the present case. On close examination 
of the plot, one quickly realizes that the overall magnitude response of S is very 
small compared to either the nominal plant or actual plant. In this case, we see 
a dip of about 25 dB. In fact, as long as the nominal plant is a reasonably good 
estimate of the actual plant, the resultant S is going to be small in gain. 

In fact, it turns out that a good gauge of S, as to its significant complexity, is 
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FIGURE 4.3. Magnitude/phase plots for S and a second order approximai 
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FIGURE 4.4. Magnitude/phase plots for M and M(S) 
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whether the nominal controller can robustly stabilize both G and G(S). In the 
event that K can stabilize G and G(S) simultaneously, then S is likely to be a 
small gain object which can then be approximated without significant loss by a 
low complexity object, denoted S. Figure 4.3 shows S, and its approximation S by 
a second order transfer function. The magnitude/phase response of M and M(S) 
of (4.9) is shown in Figure 4.4. Recall that M and M (S) are frequency-shaping 
transfer functions for (G(S) — G ), which gives us S. 

To give a better feel of what may arise in practice, let us consider another per- 
turbation of the nominal plant G, giving rise to a new plant G(S), parameterized 
by S. The magnitude/phase plot for the new G(S ) is given in Figure 4.5 alongside 
G and the corresponding S. Notice that for this case, there is a poor approxima- 
tion of G(S ) by G, so that the nominal controller K no longer stabilizes G(S). 
Note that the new S is no longer a small gain object. It is no longer possible to 
ignore the dips in the frequency response and use a low order transfer function to 
approximate S. 




Robust Stabilization 

In this subsection, we consider the class of plants G(S), S e RH^ stabilizable 
by a controller K . Let us also consider the class of controllers K(Q), Q e RH X , 
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parameterized as in (2.5.2) of the previous chapter, such that (K(Q), G ) is a stabi- 
lizing pair. We investigate the stability of the closed-loop system formed by G(S ) 
and K(Q). We have the following result: 

Theorem 4.2. Let ( G , K ) be a stabilizing plant controller pair. Let ( G , K ) have 
coprime factor representations as in (4.1) and (4.2). Consider 

G(S) = (N+ V S)(M + US )~ 1 — (M + SU)~\N + SV ), 

and 

K(Q) = (U + MQ)(V + NQ)~ l = (V + QN) '(0 + QM). 

with Q, S e R p . The pair (G(S), K(Q)) is stabilizing if and only if the pair 
( Q , S ) is stabilizing as depicted in Figure 4.6. In particular: 




Proof. Equation (4.18) is derived as follows. 




Finally using the double Bezout identity: 




(4.19) 
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( G(S ), K (Q)) is stabilizing if and only if (Q, S ) is stabilizing. This establishes 
the result. □ 



Actually, the situation of the above Theorem 4.2 can be viewed by combining 
Figures 4.1 and 2.5.1. A little effort shows that the block consisting of J and Jg 
has a transform [ ® o ] • 

This Theorem 4.2 is at the heart of the iterated designs to be discussed in later 
chapters. It leads to the following idea. An initial controller K — K( 0) stabilizing 
both model G(0) and plant G(S) can be refined by identifying a new model G(S \ ) 
and selecting an appropriate Q\ stabilizing .S'i to yield K(Q\).\n order to exploit 
the idea it is important to be able to identify S. 



Closed-loop S Interpretation 

We consider the closed-loop system G(S), K(Q ) and show how S can be inter- 
preted in terms of signals that can be obtained from the closed loop. This is an 
essential step in presenting an identification scheme for S, which is postponed 
until Chapter 5, where iterated designs are discussed. 

The following result is crucial. Refer to Figure 4.8. 

Lemma 4.3. With reference to Figure 4.8, let (G, K) be a stabilizing pair. Let 
G(S ) represent any plant stabilizable by K; (see (4.1), (4.2) and (4.3) for a pa- 
rameterization). The transfer function block J is given by (2.5.10) repeated as 



y-t _y-ijv 

The system W with inputs (vj\ , u>2,s) and outputs (e \ , 62, r ) has a stable transfer 




(a) 



(b) 



FIGURE 4.8. Closed-loop transfer function 
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function: 





r ; -/fr 1 


[M(5)l 


w = 


l~G(S) I J 






[n(S) M(S)] 


s 



(4.22) 



In particular, the transfer function from signal s to signal r is S. Moreover, the 
systems (G(S). J, Q) and (W, Q) of Figure 4.8 are internally stable. 



Proof. From Figure 4.8, simple manipulations show that W is given by 



r Ai Ai K 


r Aiy-> 


|_A 2 G(S) a 2 J 


[A 2 G(S)y- 1 _ 



V-'Ai] V~ l A 2 G(S)V~ l - V^ivj 

Ai = (7 — KG(S))- ] and A 2 = (7 — G(S)K)~ l . Now utilizing the double 
Bezout identity (4.2) and (4.1) we have 

y _1 (7 - G(S)K)~ 1 G(S)v~ 1 - y _1 A = N(S)v~ l - y _1 A 

= (N + SV)V~ l - V~ l N (4.23) 
= S , 



and the other identifications to yield the desired result (4.22). The stability results 
follow from the definition of closed-loop stability, and closed-loop stability results 
of Chapter 2. (The reader can check the details). □ 

An immediate implication of the lemma is that the (matrix) transfer function 
from the input s to the output r is S. In fact information about S can be deduced 
by observing the signals r and s. This leads naturally to an identification problem 
where the uncertainty S can be directly identified through observations of r and s. 
This is developed in Chapter 5, but suffice it to say here, for the earlier ARMAX 
example (4.12)-(4.17), that we have 

$(z~ l )r = s (z~ l )s+ s(z~ l )w. (4.24) 



Generalizations of the above results for the case of the situation depicted in Fig- 
ure 4.9 and denoted (P(S), J, Q ) are straightforward. They are also a natural 
generalization of the results developed for the scheme ( P , J, Q ) of Figure 2.5.5. 
Indeed, as expected, 



\Tu(S) r 12 (S)l 
' 1/21 CS) 722(5) J 

r 7>n(S) + Pn(S)UM(S)Pn(S) />i 2 (S)M(S)1 
' [ M(S)P 2 i(S) S J ' 
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FIGURE 4.9. Plant/noise model 

We see that the arrangements of Figure 4.9 can be viewed as T ( S ) in feedback 
with a controller [o g]> since there is zero feedback from e to w. Thus closed- 
loop internal stability of the systems of Figure 4.9 requires the stability of the pair 
([ 0 Q ] > T (&))• This is a stronger condition than merely requiring that (Q, S) is 
stabilizing. 

That stability of the pair ([ o q ] ■* T (5)) is also a sufficient condition for inter- 
nal stability of the systems of Figure 4.9 and follows by exploiting the necessary 
condition that ( Q , S) is stabilizing and thus (K (Q), G(S )) is stabilizing. First ob- 
serve that the responses r,s,e to bounded input w are bounded under the assump- 
tion, which means that y = Vr + VNs is also bounded. Now with (K(Q), G(S )) 
stabilizing, bounded disturbances at y give rise to bounded response to u, so that 
the disturbance w gives rise to bounded responses r,s,y,u,e. It is not difficult 
to see that bounded disturbances in r, s, y, u, w give bounded responses in e, and 
stability of each of the systems of Figure 4.8 imply the stability of the other. 



Plants Regulated by a Controller 

As we have seen already in the previous chapter, regulators are essentially sta- 
bilizing controllers for plants augmented by the deterministic disturbance class 
model, see (2.1) - (2.6). The augmentations can be absorbed in P, and indeed 
P(S). This allows us to apply the theory for the class of all plants stabilized by a 
controller, and thereby deduce for the unaugmented P, or P(S), the appropriate 
results for the class of plants regulated by a controller. We do not proceed further 
to spell out details here. However, we should stress that if the class of distur- 
bances is itself uncertain, and therefore S dependent, then the augmentations will 
be also S dependent. This in turn will raise concerns about the existence of any 
finite dimensional controller to achieve the regulation. For certain nongeneric S, 
say belonging to a discrete set, it may be possible for the pair ([ o q ] . T (5)) to 
be stabilizing for some Q. Here T ( S ) in (4.25) is now the plant augmented with 
the disturbance model. 

In dealing with the question of robust regulators it is usual to assume that 
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the class of disturbances is precisely known and thus not S dependent. Adaptive 
schemes, as developed in Chapter 6, overcome this problem to some extent. 



Main Points of Section 

In this section a special class of frequency-shaped uncertainty is introduced. The 
uncertainty is characterized by a (matrix) transfer function S which, when re- 
stricted to RHoq, also parameterizes the class of all plants stabilizable by a con- 
troller. The (matrix) transfer function S turns out to be the deviation of the actual 
plant from a nominal plant in the class, frequency- shaped to emphasize the op- 
erating frequency band of interest. Robust stabilization results arising from the 
parameterization in term of S are also presented. It is demonstrated that the (ma- 
trix) transfer function S can be accessed via signals measurable in the closed-loop 
system. Finally, we point out the relevance of the results to characterize the class 
of plants with deterministic disturbances regulated by a controller. Stabilization 
theory is applied to plants augmented by the disturbance models. 



3.5 State Space Representation * 

In this section, we consider again all plants stabilized by a given controller K, 
denoted G(S), where S is an arbitrary stable proper rational transfer function. The 
results presented are at a more advanced level than most of the book. They are of 
interest in their own right but the casual reader need only see that deeper results 
do exist in these waters. In particular, we are interested in providing convenient 
state space representations for any plant G expressed as G(S) where G = G( 0) 
is a nominal model and S is a parameterization. The question of minimizing the 
order of realizations is addressed. A constructive approach for any plant G and a 
nominal model G to achieve a minimal degree parameterization S is presented. 

The material in this section is used in the analysis of the adaptive controller 
based on the Q -parameterization in Chapter 6. 

Consider a controller K = UV~ l = V~ l U with U, V, U, V e RH a 0 and a 
nominal plant G = NM~ l — M~ l N with M, N, M, N e RH 0 0 . Let the Bezout 
identity (4.2) be satisfied. We consider first how a state space realization can be 
devised for G in the form G(S) in terms of the realizations: 





A 


B 1 


b 2 


v \ M U ~\ 


— 








Ci 


Du 


D\2 




. c 2 


£>21 


£>22 _ 



^This section may be omitted on first reading, or at least the proofs which are relatively technical. 
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and 





Bs ' 


Cs 


D s . 



Here Z is referred to as a stable linear fractional representation of G. Assume 
that A := (Du + DnDg) -1 exists. We can then represent G = G(S) as G(S ) = 
(N + VS)(M + US)~ l , where: 







A 


BiCs 


B\ + B2DS 


[m + us] 


\m t/lpl. 


0 


A S 


Bs 


[a + vs\ ” 


-[« vJUJ- 


Cl 


DnCs 


D 11 + D12DS 






C2 


T>2lCs 


D 21 + D22DS 



Manipulations using the techniques of Chapter 2 tell us that G, here G(S), has 
realization 





A — f2iACi 


B2CS — S2i AD12CS 


S2i A " 


G : 


-BsACx 


A s — B s ADi2Cs 


B s A 




— f^2ACi + C2 


— Q.2AD\2Cs + D22CS 


n 2 A _ 



where £2i = B\ + B2Dg and ^2 = D21 + D22D5. Given this realization of 
G = G(S), we have the following lemma. 

Lemma 5.1. Consider the state space realization for Z, S and G — G(S) in 
(5. 1)— (5.3). Let and D n + D\ 2 D S be invertible. Let S of (5.2) be a 

minimal realization. Then any uncontrollable modes of G as realized in (5.3) are 
also poles ofZ, and any unobservable modes ofG of (5.3) are also poles ofZ~ ] . 



Proof. Let aq be an uncontrollable mode of G. Then noting (5.3), there exists 
nonzero vector x' = [f f] such that 

Lol — A + ACi — B2CS + £ 2 i AD12C5 

B s ACi Lol — A s + BgADuCs B s A 
Post multiplying with the invertible matrix: 





I 0 O' 
0/0 
-Ci -D l2 C s A" 1 



yields the equivalent expression: 



-B 2 C s 
x 0 l - As 



B\ + B 2 D s _ 0 
Bs \~ 



(5.4) 
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This implies that x\ (XqI — A) = 0. Since x 0, we can claim that X] / 0 be- 
cause, otherwise, (5.4) would result in x' 2 [ XqI-As B s ] =0, which is contradictory 
to the assumption that (As. Bs ) is controllable. In this way, it follows that Ao is an 
eigenvalue of A. Also, if Ay is an uncontrollable mode of multiplicity k, then there 
exist k linearly independent vectors x. = [ x' n x' i2 i = 1 , k, satisfying (5.4), 
which in turn tells us that x;i, • • • , Xki are linearly independent eigenvectors of A 
associated with Ay. Hence, uncontrollable modes of G are poles of Z as claimed. 

Now observe that the inverse of T := D22 — ( D21 + O22O5) AD12 exists as 
the (2, 2) -block element of the block matrix 

\Dn + D l 2 D s £> 12 ! 

\_D21 + D 2 2D S O22J 

Thus, denoting As — As — BsAD^Cs. and again working with (5.3), the matrix 



A .1 — A + (B 1 + B2Ds)AC\ ( B\ + B 2 Ds)ADi2C s — B2CS 

B s ACi XI - A s 

C2 — (D21 + D 22 Ds)AC\ D22CS — (D21 + D 22 Ds)AD\ 2 Cs. 



is equivalent to 



XI — A + ACi + [Z?2 — AD\2]T l [C2 — S22AC1] 
B s ACi 

r- x [C2- ^ 2 aCi] 



0 

XI -As 
Cs . 



which in turn can be reorganized as 



u-a+[ b , *][£; 

B s ACi 

T~ l [C 2 - ^ 2 ACi] 



0 



XI -A s 
Cs . 



(5.6) 



In view of the equivalence between (5.5) and (5.6), and the fact that the poles of 
Z _1 consist of eigenvalues of the matrix 



~D n 


Dl2~\ 


1 r ci " 


D21 


D2i\ 


C 2 _ 



one can show that the unobservable poles of G are poles of Z 1 , using the same 
argument concerning uncontrollable modes of G being poles of Z. □ 



We can now prove the following lemma which is of interest in relating the 
orders of a given plant G(s) to the order of S and Z. 
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Lemma 5.2. With the same notation and hypotheses as in Lemma 5.1, and let C s 
be full row rank: 

1. The state-space realization (5.3) ofG = G ( S ) is stabilizable and detectable 
(with no unobservable or uncontrollable unstable modes). 



2. Let <5(-) denote the McMillan degree (the degree of a minimal state realiza- 
tion), and m the number of rows of C. Assume that (A,C i) is observable 
and that 



|_ Ci Du 

Then there holds 



°u 

Dnj 



for alike . (5.7) 



S(G) = S(Z) + S(S), 



(5.8) 



Proof. Item 1 is a direct consequence of Lemma 5.1 since both Z and Z -1 are 
stable. 

To prove Item 2, observe from Lemma 5.1 that (5.8) holds if the realization 
(5.3) does not contain unobservable or uncontrollable modes. Write the state ma- 
trix of the realization (5.3) in the form 



A — (B\ + B2Ds)AC\ B2Cs — ( B\ + B 2 Ds)AZ>i 2 Cs 
I — BgACi As — BsADnCs I 

[A — fijACi -BiADnCsl 



A 



0 



0 



J 



([' 



[" -B 2 ol \D S CsITaCi -AD 12 Cs~] 

[ 0 [Bs A s \ [ 0 I \ ' 

1, Ci) implies that of 

]■[? - A rf 



It is easy to see that the observability of (A, Ci) implies that of 



A — B\AC\ -BiADnCs 
0 0 



It will be shown that the pair 



^A-BiACi -BiADi 2 C s j B 2 0~^ 

is controllable for generic (Cs, Dg). In fact, the pair (5.10) is controllable if and 
only if 

rank [a./ - A + BiACi BiAD n Cs B 2 ] = 5(G), for all k e , (5.11) 
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which is equivalent to 






-B 2 

0 



0 



DnCs 



-s, 1 

£>n + DnDsj 



for all X e , 
(5.12) 



under the constraint that Du + is invertible. Because 5(G) = S(Z) and 

the fact that Cs has full rank, (5.12) follows (5.7). Thus from Davison and Wang 
(1989), the left hand side in (5.9), regarded as a closed-loop state matrix under a 
static output feedback, has in the generic case no uncontrollable or unobservable 
modes. □ 



Item 2 of Lemma 5.2 tells us that given a Z satisfying certain properties, the 
order of a plant G = G(S) generated via this Z generically equals the sum of the 
orders of S and Z. For a high order plant, here denoted G = G(S), it is often 
convenient to work first with its model, here denoted G(0) and then with the 
frequency-shaped modeling error S. This method can divide a complex design 
problem into two or more simpler problems. Needless to say, the choice of model 
is important for success of the method in terms of efficiency and computational 
reduction. The acceptable choice should be one resulting in S which has a lower 
order or can be approximated by some S of low order. Obviously, the most ideal 
case is where the complexity of S is the plant’s complexity minus the model’s 
complexity. This motivates a question as to whether there exists a model for a 
given plant such that the order of the plant is the sum of the orders of the model 
G and of S and how such G can be constructed if it exists. To address this issue, 
we need to consider a minimal stable linear fractional representation for a plant, 
whose definition is given as follows. 

Definition. In the notation above, a plant G is said to have a minimal stable linear 
fractional representation if there exists a stable linear fractional representation Z 
belonging to the set of all such representations for a nominal plant G, and an 
associated system S e R p such that G = G(S) and 

S(Z),S(S)>0 and 8(G) = 8(Z) + 8(S). (5.13) 

It turns out that the problem of existence of such minimal representations for 
a given plant is closely related to the problem of the existence of a minimal 
factorization of a transfer function. The latter problem has been addressed and 
solved, see Bart, Gohberg, Kaashoek and Dooren (1980), Dooren and Dewilde 
(1981). Recall that the factorization R — R\R 2 is said to be minimal if 8(R) — 
8(Ri) +8(R 2 ) where R, R\, R 2 are square rational matrices. We proceed through 
this deeper theory as follows. 

From Wonham (1985) we recall the following definition: 

Definition. Consider [ ^-|-§]. Denote the state space by X. Also, 4> c X is called 
a stable invariant subspace for the pair (A, B) provided (A + B F)<t> c <t> for some 
F such that (A + BF) is stable. 
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The following two lemmas are needed in the proof of the main result in this 
section. 

Lemma 5.3. Given a nominal plant G e R p . Then Z = y ] with Z, Z -1 e 
R Hoc satisfies 



G = NM~ l 
8(G) = 8(Z ) 



(5.14) 



if and only if there exists a minimal realization of G together with a sta- 

bilizing state feedback gain F and a stabilizing output injection H such that 



' A + BF 


B -H ' 


F 


I 0 


_ C + DF 


D I 



Proof. The if part was established in Section 2.4. In particular compare with 
(2.4.18). Now it is assumed that the block matrix Z satisfies the conditions (5.14) 
and has the following minimal realization 





A 


Bi 


B 2 ' 


Z : 


Ci 


I 


0 




. c 2 


D 


I 



By definition of the coprime factorization, Z and Z 1 e RH 0 c , then A and A — 
B\C\ — B2C2 + B2DC1 are stable. Setting 

A — A — B 1 C 1 , B — B\, C = C2 — DC\, 

F = Ci, H = -B 2 , 1 

one can check that [£f§] is a minimal realization of G, that F is a stabilizing 
state feedback gain and H a stabilizing output injection, and that (5.15) holds. □ 

Note that for Z given in (5.15), the conditions for (5.8) to hold generically 
amounts to the minimality of the system (F, A, H). 

The following lemma is proved in Bart et al. (1980) or Dooren and Dewilde 
(1981) and the interested reader is referred to the papers for the proof. 

Lemma 5.4. Consider an n x n invertible matrix transfer function G with mini- 
mal realization: 



G : 



A B 
C D 



(5.18) 
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Then there exists a minimal factorization for G if there exist independent sub- 
spaces X\ and X 2 such that 

AX i c Xi, 

(A - BD~ l C)X 2 C X 2 , (5.19) 

Xi @X 2 = X, 

where X denotes the state space, and ® denotes the direct sum. 

Theorem 5.5. A given plant G e R p with a minimal realization (5.18) has a min- 
imal stable linear fractional representation Z if and only if there exist two stable 
invariant subspaces X \ and Xi associated with (A. B) and {A' , C' ), respectively, 
such that 



X x ® Xf = X. (5.20) 

where X denotes the state space of the realization (5.18) and Y 1 - denotes the 
orthogonal space ofY. 



1. Necessity. 

Assume that there exists a nontrivial Zo = [ y°] e R N 0 o with Z ( y 1 € 

RH 0 o and a system S e R p such that 

G = (No + VoS)(Mo + UoSy 1 and 8(G) = S(Z 0 ) + S(S). (5.21) 

Since one can associate S with a unit§ Z 5 = [ ^ ] where N sMf 1 being 

an right coprime factorization of S and 8(Zs) = <5 (5), it is seen that G has 
the following right coprime factorization 

G = (N 0 M S + V 0 N S )(M 0 M S + U 0 N S ) '. (5.22) 

Dehne 



~M f/1 


_ ~M 0 


Uo 


~M S 


U S ~\ 


N Vj " 


_N 0 


Vo. 


_N S 


VsJ 



Since 8(Z) < 8(Z 0 ) + 8(Z S ) = 8(G) and 8(Z) > 8(G), it follows that Z 
satisfies (5.14) and 



8(Z) = 8(Z 0 ) + 8(Z S ). (5.24) 



§ We say that Z e RH^ is a unit if also Z~ l e RH 0 
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Therefore, Z = ZoZj is a minimal factorization of Z. By Lemma 5.3, there 
exists a minimal realization [— |-^] of G, a stabilizing state feedback gain 
F and a stabilizing output injection FI such that 



+ ~F 

F I 
+ " F 



-H 



0 



(5.25) 



is a minimal realization of Z. By Lemma 5.4, there exist subspaces Yi and 
I2 such that 



(a) ( ” + ~ F)Y\ C Y\ 

(b) C +H')Y 2 CY 2 (5.26) 

(c) Li ® Y 2 = X 

Evidently, (a) above implies that Y\ is a stable invariant subspace of ( , ) 

while (b) implies that Y^ is that of ( '). Since ( , , , D) and 

(A, B, C, D) are minimal realizations of G, there exists a similarity trans- 
formation T such that 

" =T~ l AT, ~ =T~ 1 B, " =CT. 

Let X\ = TY\ and X 2 = (T -1 )'^- Then it is not hard to see that X\ and 
X 2 are two stable invariant subspaces X\ and X 2 associated with (A, B) 
and (A', C'), respectively, and satisfy (5.20). 

2. Sufficiency. 

Choose F and H such that A + BF and A + HC are stable, and that 

(A + BF)X 1 C Xi , (A' + C'H')X 2 C X 2 , (5.27) 

where X\ © Xj = X. By Lemma 5.4 this implies that the Z with the min- 
imal realization (5.15) has a minimal factorization, say Z = Z1Z2 where 
Z, £ R p . Since Z is a unit in RH 0 Q and there is no pole/zero cancellation 
between its two factors Z\ and Z2, Z\ and Z2 are units in RH 0 0 . Fur- 
thermore, there is no loss of generality in assuming that Z,-, i = 1.2 are 
representations of G. Letting S := N 2 ALT 1 leads to G = G(S) derived 
using a model G\ with factorization matrix Z\, which implies 

5(G) < 5(Z X ) + 8(S) < 5(Z|) + 5(Z 2 ). 

From the minimality of the factorization Z = Z1Z2 and the above inequal- 
ities, it follows that 5(G) = 5(Zi) + 5(5). In this way, the proof of the 
theorem is completed. 
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□ 

Corollary 5.6. With the same assumption and notation as in Theorem 5.5. If 
{A, B) and (A', C') have a common stable invariant subspace, then G has a min- 
imal stable linear fractional representation Z. 



Construction of minimal Z 

The proof of Theorem 5.5 apparently provides a two-step procedure to construct 
a minimal Z. The first step is to find a pair of stable invariant subspaces X\ and 
X2 of (A, B) and (A', C'), respectively, which satisfy (5.20). In the single-input, 
single-output case, this reduces to finding a nontrivial stable invariant subspace 
of (A, B) since (A, B) and (A', C') have the same set of stable invariant sub- 
spaces. An iterative algorithm to compute the supremal (A, B)-stable invariant 
subspace contained in a subspace is given in Wonham (1985). The second step in- 
volves constructing a minimal factorization of a representation Z. For algorithms 
to perform a minimal factorization of a rational matrix, see Dooren and Dewilde 
(1981). 



Main Points of Section 

In this section, we begin with a state-space representation of a stable linear frac- 
tional representation Z and show where pole/zero cancellations may occur for 
the state-space realization of the matrix transfer function G{S) given Z. We then 
move on to derive a generic McMillan degree relation between the matrix trans- 
fer function of the plant G(S) and the stable linear fractional representation Z 
derived based on the nominal representation G and S. 

Finally, we consider the problem of minimal representation of any plant as 
a stable linear fractional representation of another plant. In particular, given an 
actual plant G, we derive a nominal model G and an S such that the order of G 
equals the sum of the order of Z = [ ^ y ], derived from G, and that of S. We 
also show that the necessary and sufficient conditions for solution to this problem 
can be derived in terms of (A, 6 (-stable invariant subspaces. 



3.6 Notes and References 

The material on signals, disturbances, disturbance responses, and their norms as 
performance measures is now quite standard in the optimal control literature. 
Likewise, the use of norms in both the time domain and the frequency domain 
to express plant performance and plant uncertainty is now standard in books fo- 
cusing on robust control issues. For a parallel treatment see for example Boyd and 
Barratt (1991) and Green and Limebeer (1994). 
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The characterization of the class of plants G(S) stabilized by a controller, pa- 
rameterized in terms of an arbitrary S € RHoq is merely the dual of the stabi- 
lizing controller class K(Q) parameterized in terms of arbitrary Q e RH^, see 
Chapter 2. The robust stabilization theory for the pair (K (Q). G(S )) was first 
developed in Tay, Moore and Horowitz (1989). Its generalization to plants P(S ) 
incorporating a disturbance response, that is to pair ( K(Q ), P(S)), is straightfor- 
ward, as is the application of the results to robust regulation. 

The main lemmas and theorems concerning state space representations for 
plants, and their nominal representations and uncertainties S are taken from Yan 
and Moore (1992). 

Problems 

1. Computing norms is not a trivial exercise. For the scalar signal dk = 

sin(«k), compute the following norms: || || | . | , || || p , rms value. How 

do the different norms compare? 

2. Show that || m || rrns < *Jn WuW^ for a vector valued signal 

u = {u k \k= 1,2, ...,u k € "}. (6.1) 

3. For scalar sequences {«&, k = 1, 2, . . . uk e } consider: 

1 N 

\\u\\ aa = lim — 'Y'lukl. (6.2) 

N^-oo Jy * — j 

Compare ||u|| aa with llwllm^. Is \\u\\ aa a norm? 

4. Show that the rms value of a signal u, such that uk = 0, is zero. 

5. Given 




find a stabilizing controller K for the model G that also stabilizes G. Ex- 
press A" as a function of a. Express G = G(S) with respect to this con- 
troller. Discuss (Q, S) and (G(S), K(Q )) in terms of a. 

6. From Lemma 4.3 verify that the relationship between r, s and W \ , u>2 of 
Figure 4.8 is given from 

r = Ss + M(S)w 2 + N(S)w\. 



(6.3) 




CHAPTER 4 



Off-line Controller Design 



4.1 Introduction 

In Chapter 2, the parameterization of a stabilizing controller K(Q ) for a lin- 
ear plant in terms of a stable (matrix) transfer function Q is discussed. It is 
shown that if Q spans the class of all stable proper (matrix) transfer func- 
tions, the entire class of stabilizing controllers for the plant is spanned. It can 
make sense in a controller design to optimize engineering objectives over this 
class, rather than over the class of all possible controllers, which of course in- 
cludes the undesirable subclass of destabilizing controllers. In this chapter, we 
present various off-line optimal controller designs that, in addition to stabi- 
lization of the nominal plant will also allow the controller to track some ref- 
erence signals and/or reject various classes of disturbances in some optimal 
fashion. 

Before going into design methodologies, it is necessary to discuss performance 
criteria for optimization. Different applications have different control require- 
ments. High performance control demanded for some feedback loops may not be 
important in other loops. As an example, in an oil refinery, typically less than 20% 
of the loops are critical and require well-tuned controllers. Many of the control 
loops are applied to buffer tanks where there is a less demanding control objec- 
tive; simply to ensure that the tanks do not overflow or become empty. Of course, 
there is possibly scope to reduce the size of the tanks and thus refinery costs by 
improving control strategies. 

For control loops where performance is relatively important, the control ob- 
jective is usually to keep some or all the states close to desired references in 
the presence of disturbances. In regulation problems the references are constants, 
whereas in tracking problems the references may vary with time. The desire to 
keep the states close to the references is obviously justified. However what is not 
yet precise is the meaning of ‘close’ . We caution that such is frequently debatable 
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and is problem dependent. 

There are many ways to specify the objective to be achieved by a controller. 
For less critical loops, this may be a visual inspection of how closely the states of 
the process follow the references. For more critical loops, specification becomes 
more precise. Performance requirements can be prescribed in the time domain, 
the frequency domain or both. 

In this chapter we will first discuss some useful performance indices and the 
sort of situations where each of these indices is commonly used. We will then 
present some off-line controller design techniques that allow us to select a con- 
troller which minimizes these performance indices. 



4.2 Selection of Performance Index 

One of the first tasks in the design of a controller is to specify the performance to 
be achieved by the control system. Specifications can be in the frequency domain 
or the time domain. For the frequency domain approach, there are specifications 
on the gain and phase over the pass- and stop-band of the desired closed-loop 
system. For the time domain approach, there are specifications on the output be- 
havior of the closed-loop system for given input sequences. The input sequences 
can have many forms. They can be deterministic signals such as constants, steps, 
ramps, sinusoids of known frequency or other known signals. Alternatively, they 
can be stochastic signals where only certain signal statistics are known. Exam- 
ples of stochastic signal statistics are mean energy level, signal level variance and 
spectrum. 

In classical controller design, the reference input is usually a known determin- 
istic signal, such as the unit step function. In this case, the performance of the 
controller is specified in terms of the closed-loop process output response to the 
unit step, commonly just known as the step response. 

In modern controller design where a linear dynamical model based approach is 
adopted, any known deterministic input signal is usually modeled as the output of 
an autonomous dynamical system with appropriate initial conditions. The resul- 
tant model of the plant in this case is then an augmented model which includes 
dynamics to generate the deterministic input signal from appropriate initial condi- 
tions. The composite model is constructed so that the tracking error is an output of 
this model. A significant area for modem controller design is for tracking or reg- 
ulation performance when the inputs are stochastic in nature and therefore cannot 
be modeled in this way. One can use a stochastic model such as a linear dynam- 
ical system driven by noise, as for example white noise. In this case, a controller 
is designed to minimize the norm of the error signal between some desired tra- 
jectory and the process output. Let us first recall for the sake of completeness a 
typical step response specification in classical controller design before moving on 
to examine error signal specification. 
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Step Response Specification 

Reference signals that remain constant for a long period of time and change to a 
new value in a relatively short period of time are usually represented for control 
design purposes as a unit step. 

The steady-state closed-loop unit step response of the process output is usually 
specified by an asymptotic tracking requirement written as follows 

lim yk — 1. (2.1) 

k-r oo 

This condition requires that in steady state there be no offset of the process output 
yk from the reference signal. That is, the difference between yk and the constant 
reference must eventually reach zero. 

The transient response on the other hand is commonly specified by some or 
all of six attributes; namely delay time, rise time, peak time, maximum overshoot, 
maximum undershoot and settling time. These six attributes are defined as follows. 

1. Delay time t ( f. This is the time required for the response to first reach 50% 
of the final value. 

2. Rise time t r : This is the time required for the response to rise from (say) 
10% to 90%, or 5% to 95%, or 0% to 100% of the final value. For under- 
damped second order systems, the time from 0% to 100% is used. For over- 
damped systems, the time taken from 10% to 90% is used. 

3. Peak time t p : This is the time taken for the response to reach the first peak 
of the overshoot. 

4. Maximum overshoot y over \ This is the maximum value of the response 
curve measured from unity defined as follows: 

y over =max(y* - 1). (2.2) 

5. Maximum undershoot y under \ This is the maximum negative value of the 
response curve defined as follows: 

y under = ma x(-y k ). (2.3) 

k> 0 

6. Settling time t s : This is the time required for the response curve to reach 
and stay within a range of certain percentage (usually 5% or 2%) of the 
final value. 

The various attributes are illustrated in Figure 2.1. It is noted here that some of 
the above specifications may conflict with one another. 

Another requirement that is specified alongside the above step response specifi- 
cation are constraints on the actuator. In any practical control system, the size and 
frequency of the actuator signal or the output signal of the controller is effectively 
limited by saturation or mechanical limitations of the actuator. 
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Error Signal Specifications 

Let us consider the system of Figure 2.3.1, with (2.3.2) reproduced here for con- 
venience: 



e = P n w + P\iu, 

y = Pnu> + P22U, (2.4) 

u = Ky. 

Let Wk and ep denote the value of the vectors w and e at the kth sample time. 
Here the input vector sequence w includes both disturbances into the system of 
Figure 2.3.1, and reference signals for the system outputs to track. We will exclude 
here known deterministic reference signals which can be modeled and included 
into the plant model. The vector sequence e is the system response to control sig- 
nals u, and to disturbances and reference signals w . The vector e is constructed by 
selecting appropriate P\ \ and P\2 to form components consisting of appropriately 
scaled functions or filtered values of the states of P22 and the input w. The two 
vectors can be written into their component form as 

Wk = [«** w 2 ,k ■ ■ ■ u>p,k\ . e k = [ei.jt e 2 ,k ■ ■ ■ <w] . (2.5) 

We do not explore in much more detail here the process of the Fn, P12 selection. 
Sometimes this is obvious given the design objectives, but at other times, the se- 
lection may be the most important part of the control design and require a number 
of trials. There is no ‘optimal’ method for the P\\, P\2 selection. This is where 
it can be an advantage to think in classical frequency domain terms as we now 
briefly discuss. 
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We know that (first or second order) minimum phase systems are the easiest 
to control and allow robust, good performance designs. The frequency shaping in 
Pn, P\ 2 is often selected so that e looks like the output of such a system. 

Also in classical control design, we are aware that in closing a control loop 
and increasing the loop gain, the closed-loop poles move to the open-loop zeros 
where these exist and to infinite asymptotes otherwise. There is difficulty in clas- 
sical design to achieve a robust high loop gain design when the open-loop zeros 
are ‘unstable’ or nearly so, since these attract the closed-loop poles as the loop 
gain increases. The classical proportional, plus integral plus differential (PID) 
controller design technique effectively introduces open-loop zeros in appropriate 
locations so that in closed loop, as the loop gain increases, the poles approach 
these assigned locations and a desired system bandwidth determined by these is 
achieved. Here the selection of P\2 can be such as to introduce zeros in desired 
closed-loop pole locations. That is, the insights of PID design can be brought to 
bear in designing Pi 2. Closing the loop with the appropriate (frequency shaped) 
gains is then the province of the optimal disturbance rejection method. The op- 
timization is such as to achieve stability, desired (approximate) closed-loop pole 
locations, and thereby desired bandwidth, as well as disturbance rejection. 

With the sequence e appropriately constructed, as just discussed, the next step 
is to formulate a measure of e so that we can design a controller to minimize e 
according to this measure. The commonly used measures of e are norms such as 
the one-, two- and infinity- norm. Which norm to use will depend on the problem 
to be solved. 

The question to be asked is: If the error vector e deviates from its ideal value, 
what is the consequence to the overall objective of the control problem? To answer 
this question, we will have to examine the relationship between the controlled 
variables defined within e and the physics of the process concerned. For example, 
in chemical processes, the variables to be controlled in an operation should be 
at some particular levels prescribed by physical, chemical and thermodynamical 
laws. A deviation of any controlled variable from the stipulated level will upset 
the chemical reaction which may lead to undesirable effects. 

Specification using 2-norms 

This specification is popularized by the so-called linear quadratic (LQ) design 
method. Here we define the performance index J as 

J = llelll, (2.6) 

With the definitions expressed in (2.5), J can be written as (see Appendix B for 
2-norm definitions) 

7 = E( e U + e lk + ---+e 2 m , k ). (2.7) 

In an LQ regulation design, the error components e, represent the plant states and 
controls or a linear combination of these. In the LQ tracking case, the errors are 
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the difference between some reference signals and certain linear combinations of 
the states. Of course, without any penalty on the control energy, high performance 
can be achieved for the model, but practical designs require some trade between 
tracking (regulation) performance and control energy to achieve this performance. 

If we assume that the weights are built into the components of e, then it is 
obvious that this formulation is a weighted sum approach which combines the 
various components of the e vector into a single performance index. The weights 
need not necessarily be constants, but can be time varying. 

There is also the concept of frequency shaped weights to give more penalty 
to certain components of e in some frequency bands in relation to other bands. 
Frequency shaping filters can be actually built into the dynamics of Pn, P\ 2 just 
as can constant weights on the components of e; . A commonly used frequency 
shaped filter is the integrator given as 

F(z- 1 )=-^ t . (2.8) 

1 - z~ l 

Now F{z~ l ) has an infinite magnitude at zero frequency (z = 1). This gives 
rise to an infinite penalty for an error of zero frequency. This infinite penalty at 
zero frequency ensures that the optimal controller achieves zero steady-state error. 
Other frequency shaping biters may penalize control energy at high frequencies to 
ensure that high frequency lightly damped unmodeled dynamics are not excited. 
The concept of frequency shaping is important because it allows us to incorporate 
frequency domain specibcations into an otherwise time domain performance in- 
dex. It should be emphasized that in an engineering design, an engineer may well 
spend a large portion of the project time on selecting and adjusting frequency 
shaped biters. 

With the performance index J, as given in (2.6), the design task is then to select 
a stabilizing controller K, or equivalently using the Q parameterization approach, 
a stable matrix transfer function Q that will minimize the index J, as 




To perform this minimization, we will have to write the errors e* in term of the 
matrix transfer function Q and the input disturbances and references w. With Fq 
denoting the transfer function matrix from w to e, in terms of Q, the key relevant 
equation is (2.5.18), repeated here as 

e = F q w\ F q = (Fn + PuUMP 2l ) + P l2 MQMP 2l e RH (2.10) 

Recall that Fq is affine in Q. For the frequently considered special case where w 
is white noise, the minimization of (2.9) is then equivalent to 
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Worst Case Design using 2-norms 

Let us consider a 2-norm cousin of the index (2.6) 

7= max Hell? , (2.12) 

wei 2 ,\\w\\ 2 <l 

where the I2 space is defined in Appendix B. Here, the performance index is the 
worst case 2-norm of e over all 2-norm bounded input disturbances w. Thus w 
has bounded energy. The controller design task is then given as 



ib S . 52 C? 



(2.13) 



If the matrix transfer function from w to e is Fq of (2.10), then it turns out 
that (see Vidyasagar (1985), and Chapter 3) this minimization can be rewritten 
in terms of an oo-norm as 



min II Foil" 

QeRHoc " " c 



which is the so-called H 0 0 minimization problem. Here || Fq || ^ is defined a: 

I F sis*? = su p 



= sup CTmax (F Q {e je )\ , 
0e(-7i,n] v ' 



where er max denotes the maximal singular value. Hence, the optimization problem 
(2.13) can be restated in frequency domain terms as: 



min sup 1 

QeRHoc m 



(pQ^ 0 )) . 



The H 0 o optimal controller attempts to minimize the worst possible adverse im- 
pact of a whole class of disturbance as opposed to just working with disturbances 
and responses in an average sense as in the linear quadratic design case. In fact the 
resulting controller rejects disturbances uniformly in all frequency bands. With- 
out frequency shaping, this is not the desired outcome in most practical cases. 
The controller, though robust to the various input disturbances, generally lacks 
performance. Thus in practice, the success of an H 0 0 optimal design depends 
very much on the frequency shaping filters incorporated into the performance in- 
dex. The frequency shaping filters seek to emphasize certain frequency bands. 
Those frequency bands that contain more uncertainties are given more emphasis 
in relation to those that have less uncertainties or are outside the plant’s operating 
bandwidth. The result of incorporating these frequency shaping filters, in effect, 
is to reduce the size of the class of disturbances rejected uniformly by the H 0 0 
optimal controller. 
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Specification in oo-norm 

We have already noted a connection between 2-norms in the time domain and 
oo-norms in the frequency domain. The oo-norm specification in the time do- 
main is appropriate when we are looking at minimizing the peak of the sys- 
tem output, as it is precisely the infinity norm of the output sequence. In the 
case when the error is a vector of sequences such as e of (2.5), then its infinity 
norm is 



(2.16) 



where, with + denoting {0, 1,2,...}, 



He.-Hoo = max{|e a |}. (2.17) 

So that we can specify the nature of the optimization problem, it is neces- 
sary to be precise about which class of disturbance signals w we want to con- 
sider. If the input signal w is the class of i 2 bounded sequences (finite energy 
signals) such that ||tu||2 < 1, then (see Vidyasagar (1985) and Section 3.3) 
we have 



J — max 

JC€f2,IMl2<l 






(2.18) 



where Fq is the matrix transfer function from w to e. Moreover, it can be shown 
that the controller that minimizes this performance index is the same as the con- 
troller that minimizes (2.9) when the input w is a zero mean white noise signal of 
unit variance. 

Let us next examine input sequences that are not necessarily 2-norm bounded. 
In particular let us consider the case where the magnitudes of the inputs are 
bounded. Without loss of generality, due to linearity we write ||u)|| oc < 1. We 
can then write the following performance index: 



J= max Halloo = max IkHloo- (2.19) 

Moo<l IMIoo<U 

Again let Fq be the matrix transfer function from w to e and denote by /g = 
{f(2,k, k e +} its impulse response sequence. Then 






( 2 . 20 ) 




4.2. Selection of Performance Index 99 



The induced norm on Fq is then given as 



;=• e=o 

= m 1 ax E ;||^| 1 



( 2 . 21 ) 



with f l Q the i jth element of /q (J'q = {/g k , k e } is the impulse response se- 
quence). Thus minimizing the performance index of (2.19) turns out to be equiv- 
alent to minimizing the 1 norm of /g, or the 1-norm of Fq, the matrix transfer 
function from w to e. 



J = }&$*= Ill'll- (2-22) 

We shall next examine other variations of specifying an oo norm type perfor- 
mance index. The above specification uses a weighted maximum method to com- 
bine the various components of the vector e. There are other ways to combine the 
various components of the vector e. One possibility is to use the weighted sum 
method, reminiscent of the LQ approach, as follows. 



j = E x ‘\ 



(2.23) 



where A,- > 0 are constant weights. However it turns out that such an approach 
does not achieve a unique controller for each weight selection. Moreover, each 
optimal controller corresponds to a range of weighting selections, so that the ob- 
jective of relative penalty of the various components of e is not necessarily real- 
ized. 

Another possibility is to write the performance index as follows: 

J = Y J X i \e i \, (2.24) 



where again A,- > 0 are constant weights and e,- are the components of e. This 
differs from the previous index in that the focus is on the weighted sum of the 
magnitude of the various components at each instant. This is in contrast to the 
weighted sum of the infinity norm of each component of e. Such a performance 
index is applicable in cases where the instantaneous value of the sum of the vari- 
ous |e,- 1 is important. An example is the case where the maximum current drawn 
from a power supply at every instant of time be kept below the absolute maximum. 
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Main Points of Section 

In this section, we have discussed the formulation of performance measures. 
There are two main approaches to specify performance of a control system. The 
first, used commonly in classical controller design, is specified in terms of the 
closed-loop, steady-state and transient response to a step input. The second, used 
commonly in optimal control design, incorporates error signals into a perfor- 
mance index. The strategy to be developed here is to optimize performance over 
the class of all the stabilizing controllers for the process. 



4.3 An LQG/LTR Design 

In this section, we present the design of the very successful linear quadratic Gaus- 
sian (LQG) controller with loop transfer recovery (LTR). For further details, see 
Anderson and Moore (1989) or Kwakernaak and Sivan (1972). First, linear qua- 
dratic (LQ) controller design is based on the minimization of a quadratic perfor- 
mance index under the assumption that the plant for which it is designed is linear. 
The resultant control law is a linear feedback of the states of the plant. This con- 
troller is optimal with respect to the defined performance index and is appealing 
because irrespective of the weight selections, the closed loop is robust to certain 
plant variations. (In classical terms, continuous-time LQ designs guarantee 60° 
phase margins and [—6, oo) dB gain margins. In discrete-time no such margins 
are guaranteed!) 

In the event that the states of the plant are not accessible, then the next step is to 
replace the states by estimated states using an optimal state estimator. This gives 
rise to the LQG design. The strategy is optimal if the actual plant to be controlled 
is the nominal model used in the design of the controller. Otherwise it may be a 
poor strategy. An LQG design may have closed-loop properties indicating poor 
stability margins at the plant input and/or output, see Doyle (1974). 

A final step in the LQG based design strategy is to modify the LQG controller 
design in such a way as to achieve full or partial loop transfer recovery of the origi- 
nal state feedback design, see Doyle and Stein (1979) and Moore and Tay (1989c). 
There is usually a scalar design parameter that can be adjusted to achieve a trade- 
off between performance for the nominal design and robustness via recovery of 
the state feedback design loop gain properties. 

There are a number of concerns regarding the LQG/LTR design approach. For 
minimum phase plants, loop recovery is obtained by increasing the loop gains. 
The resulting high gain systems may not be attractive for implementation due to 
their high sensitivity to certain external disturbances, plant parameter changes and 
unmodeled dynamics. For nonminimum phase plants, full loop recovery of LQG 
designs can only take place when there is an unstable pole/zero cancellation in the 
control loop matrix transfer function and consequent instability. This suggests that 
only partial loop recovery be attempted, or that the state estimate feedback control 
laws should be constrained as discussed by Zhang and Freudenberg (1987). 
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In this section, we present an approach to loop recovery, termed sensitivity 
recovery, which is in essence a frequency-shaped loop recovery emphasizing the 
frequency region of unity gain (the cross-over frequency). Sensitivity recovery is 
achieved by augmenting the original LQG controller with an additional filter with 
matrix transfer function Q, and optimizing the Q for sensitivity recovery using 
the Q-parameterization technique introduced in Chapter 2. 



LQG Design 

Let us consider the following state space description of a discrete, linear, time- 
invariant plant as follows. 



Let 



Xk+i = Axk + B (uk + u>i,*) ; xo given, 
y k = Cx k + Du k + w 2 ,k- 



G : 



A B 
C D 



(3.2) 



where Xk e "is the state vector, e p the input vector, » e m the output 
vector and w\^ e p , W2,k e are independent white noise disturbances with 
covariance matrices given as 



[ N j N 

- ='■ Qe, lim — Y W ^kw' 2 ,k -■ R e- 



We assume that Q e = Q' e > 0 and R e — R' e > 0. 

Let us also assume that (A , B) is stabilizable and ( A , C) is detectable. Consider 
also a quadratic performance index given as follows. 

1 N 

3 =l™ 0 xY( X kQc X k +u k R c u k), (3.4) 



where Q c and R, are weighting matrices Q, = Q' c > 0 and R c = R' c > 0. In 
terms of the error signal introduced in (2.5), we have 



ek = 




(3.5) 



and the performance index of (3.4) is the squared root mean square ( rms ) value of 
this error signal. This performance index is only concerned with the steady state 
performance of the controlled loop. Indeed, any decaying transient is averaged 
out of the index (3.4). The optimal controller for the plant of (3.1) minimizing 
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the performance index of (3.4) is given as follows, see for example Anderson and 
Moore (1989): 



u k = Fx k , (3.6) 

h + 1 = Axk + Bu k + H (Cx k + Du k - y k y, *0 given, (3.7) 

where 

S c ,k — A'S C 'k+iA - A'S Ci k+iB(R c + B' S c ,k+iB)~ l B' S Ci k+iA + Q c , 

S c = k \im^S Cyk , S C , N = Q c , (3.8) 

F = ~(R C + B'S c B)~ l B'S c A, 
and 

S eM \ = AS e ,kA' - AS e<k C\R e + CS e ,kC')- l CS et kA’ + BQ e B\ 

S e = Urn^ S etk , S e , 0 = 0, (3.9) 

H = —AS e C'{R e + CS e CT l . 

Existence of S c , S e are guaranteed by stabilizability of the pah (A, B) and de- 
tectability of the pah (A, C), respectively. Closed-loop asymptotic stability fol- 
lows also from both these condition. 

The output feedback LQG controller constructed from the regulator state feed- 
back gain F and the estimator gain (output injection) H is realized as (see also 
(2.4.17)) 



’ A + BF + HC + HDF 


-H " 


F 


0 



Actually in much of the theory to follow, F and FI can represent any stabilizing 
gains for the derived plants, not necessarily obtained from an LQG design. 

Let us also define stable coprime factorizations for the plant (3.2) and LQG 
controller (3.10). In the notation of (2.4.1), (2.4.2) and (2.4.18), (2.4.19), this is 
given as 



V 

—N 



I] 

~°] 
M J 



' A + BF 


B -H 




F 


I 0 




_ C + DF 


D I 




" A + HC 


-( B + HD ) H ' 


F 


/ 


0 


C 


-D 


I 



(3.11) 



(3.12) 
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See Section 3.5 for a discussion of the minimality of these realizations. 

The class of all stabilizing controllers for the plant (3. 1) is then given by (2.5.2), 
repeated here as 



K{Q) = 


U(Q)V(Q)~ 


1 = V(Q) l U(Q). 


(3.13) 


U(Q) = 


U + MQ, 


V(Q) = V + NQ, 




U (Q) = 


U+QM, 


V(Q) = V+QN , 





and depicted in Figure 2.5.4 for the case where the nominal controller is a state 
estimate feedback controller, as for example, derived via the LQ control problem 
discussed above. 

We observe that a state space realization for normalized coprime factorizations 
for the plant G and controller K are obtained in the form (3.11), (3.12), but with 
F and H calculated from an LQG design by setting 

e =[ C, + D "]. (3.14) 

A normalized right coprime factor follows from the following control Riccati 
equation leading to a state feedback gain F : 

S c ,k = (A - BT c R-'C^j (s c ,* +l - S c , k+1 B (R c + B' S cMl B)~ l B'S c , k+ 1) , 
x (a - BT c R~ x C ) + ( Q c - T'R~ l T c ^j , 

S c = lim S c ,k, S C ,N = Qc, 

Q c = C'C, R c = D'D + 1, T c = 2 C'D, 

F = -(R c + B'S C B)~ X B'S C A , 

while the left coprime factor’s state space realization follows from the filter Ric- 
cati equation leading to an output injection H : 

S c , k+l = (A - BT e R~ x C ) (s e ,k - S e , k C' (R e + CS e , k C’)~ l CS e , k ) 

x(a- BT e R~ l C^j +B^Q e - T e R~ l T^j B r , 

1 N 

S e = lim S e k , S e o = 0, T e = lim — w\ k w' 2 
k-^oo ’ ’ 00 N ’ ’ K 

H = —AS e C' (. R e + CS e C')~ l . 

Formulation of Sensitivity Recovery 



In this subsection, we consider sensitivity functions of the full state feedback 
design and the full state estimator feedback design. We formulate an error function 
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through which we achieve recovery of loop robustness of the full state feedback 
system. As a point of notation, recall the following realizations definitions: 



Gf : 



A B 
I 0 



Gh : 



A I 
C 0 



(3.15) 



Input Sensitivity Recovery 

Consider the full state feedback control system design of Figure 3.1. The closed- 
loop matrix transfer function from w to z is the input sensitivity function matrix 
given by 



S l SF = (/ - FGf)- 1 



’ A + BF 


B " 


F 


I 



(3.16) 



For the state estimate feedback design, the input sensitivity function matrix is 
given by 

S'sef = (l - KG r 1 = MV. (3.17) 

where K is given by (3.10). Let us now consider the the class of all stabilizing 
controllers K(Q ) parameterized by Q e RH^ as given in (3.11) - (3.13). For a 
stabilizing controller K(Q), Q e RH 0 G , the associated input sensitivity function 
matrix is 



S‘ Q = (I - K(Q)G)~ l = M(V + QN). (3.18) 

(Refer to (2.5.6).) 




FIGURE 3.1. Target state feedback design 
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Let us now define an error matrix transfer function 

€ q = $SF ~ Sq. (3.19) 

When plant disturbances or uncertainties occur at the plant inputs, the minimizing 
of e l Q gives robustness to a controller design. 

Using (3.16), (3.17) and (3.18) we observe that this error matrix transfer func- 
tion is affine in Q 

e‘ Q = M - M(V + QN) = M(I - V) - MQN. (3.20) 

It can equally be rewritten as 

4 = (/ - FG f )~\fG f - K(Q)G)(I - K(Q)G)~\ (3.21) 

This allows us to interpret €q as a frequency-shaped version of the loop-gain 
transfer function error ( FG f — K(Q)G). The frequency shaping is provided by 
M = (/ — FG f )~ 1 , the target sensitivity function and (/ — K(Q)G)~ l , which is 
the actual closed-loop sensitivity function (parameterized by Q e RH a Q ). These 
weightings together serve to emphasize the unity loop gain frequencies, that is, 
the crossover frequencies. 

Output Sensitivity Recovery 

Consider the full state estimator feedback loop of Figure 3.2. The closed-loop 
matrix transfer function from w to z is the output sensitivity function matrix given 
in terms of the output injection FI as 



S° OI = (/ - G h H r 1 = M : 



’ A + HC 


H ’ 


C 


I 



(3.22) 




FIGURE 3.2. Target estimator feedback loop design 
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For the state estimate feedback design, the output sensitivity function matrix is 
given by 

S° SEF = (I - GK ) -1 = VM, (3.23) 

where K is given by (3. 10). Let us now consider a stabilizing controller K(Q) pa- 
rameterized by Q € RH 0 o as given in (2.5.2). For a stabilizing controller K(Q), 
Q € RH 0 o, the associated output sensitivity function matrix is 

S° Q = (I- GK(Q))~ l = (V + NQ)M, (3.24) 

(where we used the identity (2.5.6)). 

Introduce an error matrix transfer function as: 

e Q = S°oi ~ S°q- (3.25) 

By duality with the input sensitivity case, when disturbances or uncertainties oc- 
cur at the plant outputs, the minimization of €q gives robustness to a controller 
design. 

From (3.23) and (3.24) we observe that €q is affine in Q as follows: 

e° Q = (/ -V)M - NQM. (3.26) 

It may equally be interpreted as a frequency weighted loop gain error by rewriting 
€°q as follows: 

€°q = (I ~ GhH r 1 (G h H - GK(Q )) (/ - GK(Q))- 1 • (3.27) 

Loop Recovery via Sensitivity Recovery 

Asymptotic loop recovery of a suitably parameterized LQG design is said to occur 
when, with design parameter adjustments, the loop matrix transfer function of the 
LQG design, namely KG (or GK), approaches the loop transfer matrix transfer 
function of the target LQ (or estimator) design, namely FGf (or GhH), for all 
z = eJ (uT , 0 < ojT < it . Now since F and H are stabilizing controllers for 
Gf and Gh, respectively, and K is a stabilizing controller for G, then S l OI = 
c I-FGp)-\ S° oi {I-G h H)-\ S i SEF = (I-KG)- l tmdS° SEF (I-GK)- 1 
are well defined for all z — e^ toT , 0 < coT < n. In view of these properties, 
we see that loop recovery occurs, that is, KG —*■ FGf (GK — > GhH), if 
and only if (I - KG)~ l -+ (I - FGp) ((/ - GK)~ X (/ - G h H)~ 1 ), or 
equivalently, when the loop sensitivity function matrix of the LQG design, namely 
S l SFF (or S° SEF ), when suitably parameterized, approaches the loop sensitivity 
function matrix of the LQ (or estimator) design S l SF (or S° SF ) for all z = e^ mT , 0 < 
coT < tv. Of course, these equivalent loop recovery definitions also apply to the 
LQG design augmented with arbitrary Q e RH a c , with S l SEF and S^ EF replaced 
by Sq and S°q. More specifically, since K(Q) stabilizes G so that (I — K(Q)G)~ l 
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and (/ — GK ( Q))~ l exists for all z = 0 < coT < n, then from (3.21) and 

(3.27) we see that loop recovery occurs, equivalently, when c'q tends to zero (or 
€°q tends to zero), that is we have asymptotic sensitivity recovery. We conclude 
the following equivalent asymptotic loop and sensitivity recovery conditions. 



Lemma 3.1. Asymptotic loop recovery at the plant input (or output) occurs if 
and only if, for suitable parameterizations, there is asymptotic sensitivity recovery 
given by 

e l Q -»■ 0 (or €q-> 0) forz = e^ mT , 0 <toT <n. (3.28) 



A partial sensitivity recovery is said to occur when c'q (or €q) are made small 
in some sense. This also corresponds to a partial loop recovery, albeit frequency 
shaped by virtue of (3.21) and (3.27). Reasonable measures are the 2-norm or 
oo-norm with tasks defined as 



(" 



min e'J , 

QeRHoo II y II 2 

min e?J , 

QeRHoc II y II 2 



e sj|4IL 

eSjilL )• 



(3.29) 



These are standard Hj, H 0 Q optimization tasks (see also the next section) since 
€q, €q e RH 0 o are affine in Q e RH 0 c . When 



( or H| =0, IC=° )■ 

there is full sensitivity recovery, and by virtue of (3.21) and (3.27) there is full 
loop recovery. We have the following lemma. 



Lemma 3.2. The state estimate and residue feedback controllers K(Q), with F 
and H fixed and Q e RH 0 0 variable, achieve full loop recovery, equivalently 
input (or output) sensitivity recovery, if and only if Q is selected as Qi e RHoq 
( or Q 0 e RHoo)with 

l I-Vf-QiN , ( I-V) = NQ 0 . (3.31) 



We will next examine the existence of such <2, and Q 0 . 



Full Loop Recovery Cases 

In this subsection we will examine cases where full loop recovery can be achieved. 
Minimum Phase Plants 

We present full loop recovery results for the case of plants whose finite zeros are 
stable, that is plants with full rank properties as follows: 




is constant for |z| > 1, 



(3.32) 
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or equivalently, G is minimum phase. Moreover, we require the plants to satisfy 
the following invertibility condition: 

G~ l (left inverse) or (z,Gff L £ R p exists, (3.33) 

or 

G~ r (right inverse) or (zG)~ R G R p exists. (3.34) 

We have the following result: 

Theorem 3.3. Consider a plant (3.2) and a state estimate feedback controller 
(3.10) constructed from a state feedback gain F and a state estimate gain H and 
with Q £ RH 0 0 . Consider also associated factorizations (3.11), (3.12). Let the 
plant G be minimum phase, in that condition (3.32) holds. Full input sensitiv- 
ity (output sensitivity) recovery is possible provided the plant has a left (right) 
inverse, that is condition (3.33) (or condition (3.34)) is satisfied. The recovery 
e l Q = 0, see (3.19), (or c°q = 0, see (3.25)), is achieved when Q is selected as Qi 
( Qo) given by: 

Qi = (I — V)N~ l e RHoo (or Q a = N~ R (I - V) e RH^ ), (3.35) 

or equivalently, 

Qi = z (l - V)(zN)~ L g RH 0 o (or Q 0 = (zN)~ R z(I - V) e RH X ). 

(3.36) 

Proof. Condition (3.32), together with G = M~ l N implies that N is minimum 
phase. Moreover, if G~ h G R p exists, we have that N~ L exists, and is stable. It 
follows that condition (3.31) can be realized by selecting Q = (/ — V)N~ L . This 
selection satisfies Q G RH 0 0 by virtue of N being minimum phase. The other 
conditions can be explored in a similar manner. □ 

We remark that if the plant has the same number of inputs as outputs, the modes 
of Qi (or Qo) are identical to the set or subset of the zeros of the plant (values of 
z for which the plant G loses rank in (3.32)), and the McMillan degree is upper 
bounded by n. 

Nonminimum Phase Plants: 

As discussed earlier for nonminimum phase plants, it is in general not feasible to 
achieve full loop (sensitivity) recovery. However, for certain partial state feedback 
designs, full or partial (sensitivity) recovery can in fact be achieved. 

Qualitatively, this is done as follows. First write the plant G as a product of two 
factors where one is all-pass (possibly unstable) and the other a square minimum 
phase stable factor. Recall that all-pass systems have a flat spectrum: Poles z p 
and zeros occur in pairs satisfying z p = z.7 ] . Let us assume then that for the 
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factored plant there exists a state estimator gain and dynamic state feedback gain 
such that only the states associated with the minimum phase factor are fed back. 

For the partial state feedback, we can write down the the input sensitivity func- 
tion, and the output sensitivity function, and the corresponding sensitivity differ- 
ence functions as in (3.20) and (3.26). 

Once the sensitivity difference functions are defined, being affine in <2, (or 
Qo ), full or partial loop sensitivity recovery is achieved by appropriate selection 
of Qi and Q„, see Moore and Tay (1989c). 

Example. Consider the continuous-time, unstable, minimum phase plant with 
transfer functions 



Note that this plant has two unstable poles. Discretizing G c with a time sampling 
period of t s = 0.7, (see Chapter 9) we have for the equivalent Z-domain repre- 
sentation: 



" 4.6969 -8.1662 1 " 

G: 1 0 0 , 

_ 2.3706 -0.8809 0 _ 

with a minimum phase zero at 0.371 6 and unstable poles at 2.3484 ± j 1.6282. 

We design LQ and LQG controllers assuming Q e = I, R e = 1 and e' k = 
[ yi “4 ]. The state feedback gain and state estimator gains are given by 

F = [—4.137 5 8.0536], H' = [-2.094 9 -0.4136]. 



The Z-domain Nyquist plots for the LQ and LQG controllers are shown in Fig- 
ure 3.3. The open loop in the LQ case has two unstable (plant) poles and so en- 
circles the Nyquist point (—1,0) twice, whereas, for the LQG case, the open loop 
has four unstable poles (two plant and two controller) an so encircles the Nyquist 
point four times. Obviously, the robustness of the LQG controller is inferior com- 
pared to the LQ controller, at least in terms of gain margins and phase margins. 
Using the loop transfer recovery described in this section, we obtain <2, as 



0.371 6 
0.053 6 



1 



-1.745 



J 



e RHoo. 



This gives rise to K (Qf) which achieves full loop transfer recovery. For partial 
loop recovery, we use K (a Q ,• ) , 0 < a < 1 . Figure 3 .4 shows the extent of recov- 
ery for a = 0.5, a = 0.95. Clearly when a = 0.5 there is a significant recovery 
and when a = 0.95 there is near recovery. (Full recovery occurs when a = 1.) 
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Main Points of Section 

In this section the design of an LQG controller with loop transfer recovery or 
more precisely sensitivity recovery has been discussed. Sensitivity recovery is 
achieved by augmenting the original LQG controller with an additional matrix 
transfer function Q, feeding back the estimation residuals. We show that for min- 
imum phase plants and some nonminimum phase plants where a particular partial 
state estimate feedback controller is used, full loop recovery may be achieved. 
Otherwise only partial recovery is achieved and this can be done in an optimal 
manner through the sensitivity recovery approach using the Q parameterization. 



4.4 Hoo Optimal Design 

In this section, we present the formulas for solving the H 0 0 optimization task of 
(2.14). Most of the formulas are quoted from Green and Limebeer (1994). 



Problem Formulation 

Let us consider the plant model of (2.2.1) with realizations written as follows. 









A 


Bi 


b 2 


r 4 „ r a 


p _ |>n 


Pill . 


— 






W-'L-J 


[P21 


^22 J ' 


Ci 


Du 


Dn 








. c 2 


D 2 i 


0 



(4.1) 



In this realization*, (A, B2) is stabilizable and (A, C 2 ) is detectable. We assume 
that w is any bounded sequence with ||u>||2 < 1 and Pi 1 , P21 contain stable fre- 
quency shaping filters that determine the influence of w in the various frequency 
bands on e and y. The selection of the frequency shaping filters will depend on a 
priori knowledge of the disturbance to the plant in the various frequency bands. 
Similarly we assume that P\ \ , P\2 contain stable frequency shaping filters that 
penalize the elements of e appropriately in the various frequency bands. 

Let us consider a stabilizing controller K for(4.1) (u = Ky) and corresponding 
stable coprime factorizations of (3.13) for K and (4.1). Realizations in terms of 
the parameters in (4.1) are given in (3.10), (3.11) and (3.12) with B = B2, C = 
C2 and D = D22. We can now write down the class of all stabilizing controllers 
for (4.1) in term of a stable matrix transfer function Q £ RH^. The closed-loop 



*Without (great) loss of generality, one can assume that there is no direct feedthrough term from 
input u to the control output y. It is always possible to replace an output with feedthrough term by an 
equivalent output without it, by simple subtraction. This simplifies some of the algebraic expressions. 
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matrix transfer function is then given by (2.5.18), repeated here 



e = Fqw, 

F q = (P n + P l2 UMP 2l ) + P X2 MQMP 2 1 e RH C 
= T\\ + 7,2(2721 • 

The realization for 7 is repeated from (2.5.22) as follows. 



A + B 2 F 


-hc 2 


—HD 2 \ 


b 2 


0 


a + hc 2 


B\ + H D 2 \ 


0 


Ci + D l2 F 


Cl 


D n 


7»12 


0 


c 2 


7»21 


0 



Now consider the performance index 

J = max ||e || 2 = = l|7li + 7 i2 Galileo . 

Mh<2 

The optimization task is 



117,, +7,2(7 Tatllbc- 



(4.2) 

(4.3) 



(4.4) 



(4.5) 



(4.6) 



In order to be able to solve this so-called 77oo control problem, the following 
sufficiency assumptions are made: 

1. (A, B 2 ) is stabilizable and (A, C 2) is detectable, 

2. D' n D n > 0 and D'^Dix > 0, 

3. rank [ A ~q^ ^ j = n + m, for all 9 e [0, 27r], 



4. rank [ A -^ ! ^ ] = n + m, for all 9 e [0, 2tt]. 

Assumption 1 is obviously necessary from a control point of view; without As- 
sumption 1 no stabilizing output feedback controller can be constructed. Assump- 
tion 2 provides sufficient conditions under which the control strategy can be im- 
plemented, often > 0 will suffice. Assumptions 3 and 4 are conditions that amount 
to 7,2 and 72, having no zeros on the unit circle. These conditions are crucial as 
we need to invert 7,2 and T21 in some sense to find the optimal controls. 

Before presenting a solution to the optimal control problem (4.6), we 
present a state space formulated solution for the characterization of all controllers 
that achieve the less stringent performance objective 

l F t|k < 



(4.7) 




4.4. H 0 o Optimal Design 113 



or equivalently. 



IMI 2 < Y 



IM | 2 • 



(4.8) 



The solution to this problem requires one to solve a set of coupled algebraic Ric- 
cati equations. If for a certain y we fail to find a solution then this indicates that 
it is impossible to decrease the gain between disturbance and performance sig- 
nals any further. This observation can be used to construct a crude method for 
iteratively finding a controller that approaches the H 0 0 optimal controller that 
minimizes the criterion (4.6). 

Consider the algebraic Riccati equation: 

X = C'JC + A'XA- ' _1 , (4.9) 



where 




are defined from the plant realization matrices (see (4. 1)) and 
= D'JC + B’XA, 

= D'JD + B'XB. 

Here I\ is an identity matrix with dimension of the performance variable e, / 2 
is an identity matrix whose dimension corresponds with that of the disturbance 
variable w. The matrix 1 has row dimension equal to that of u>, and 2 has row 
dimension equal to that of the input u. Similarly for . 

Assume that the control Riccati equation (4.9) has a solution such that 





X >0 and A 


- B - 


1 is stable, 


3 > 0, 


1 - 2 


-1 

3 2 


< 0 . 


Introduce the square root factors for 


3 and - 


1 + 


2 3 1 


R'R = 3 , -y : 


l T'T = 


1- 


> -1 
2 3 


and define 








W=\ R 3_1 2 


*1-1 


Vn 


W 12 I 


L T 


°J l 


_w 2 , 


0 J 



i = z - 1 W 



(4.10) 



(4.11) 



(4.12) 

(4.13) 
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Also introduce the “bar” variables 



"A 


ft 


ft' 




' A 


- B\W 2x 'L 2 


Sift/ 


b 2 


Ci 


Dn 


D\2 


= 


Lx 


-WxxW 21 1 L 2 


ftift/ 




_ft 


ft l 


0 _ 




ft 


-D 2 xW 21 1 L 2 


£>21 ft - / 


0 _ 



Finally introduce 

•ft:} 'ft mm 

Consider now the Riccati equation 

Z = BJB' + AZA' - " ' _1 



(4.14) 



(4.15) 



' = AZC' + BJD', 



1 2 

_ 2 3_ 



= DJD' + CZC'. 



(4.16) 



(4.17) 



Assume furthermore that the filter Riccati equation (4. 15) has a solution such that 
Z > 0 and A — ~ 1 C is stable, 

S3 > 0, 1 — 2 3 2 ^ < O' 

Under the conditions in (4.10) and (4.17) there exists an output feedback con- 
troller that achieves the performance measure (4.7) or (4.8). All such controllers 
are generated from 



Xk+X 




'A C 


Bex 


Bci 


Xk 




Uk 


= 


Ccx 


Dcxx 


L>CX2 


yk 


(4.18) 


. r k 




Cci 


L>C2X 


0 


Jk_ 





where ft, sy) are any signals that satisfy a relationship 
s = Qr, 

where Q(z) is a stable rational transfer function such that ||<2lloo < Y ■ The ma- 
trices in (4.18) are defined via the solution of the coupled Riccati equations (4.9) 
and (4.15) as follows: 



A c — A — ftft/ft + [ft 1ft 1 W ] i - l] ft - / ft, 



[#o B ci ] 



77cn 

Den 




770121 



(ftlft/fti ~ ft) ft,' ( 'U ~ ft ft/ ft)] . 
ft - / (ft - ft, ft- 'ft) 
ft”/ ft 

ft ft"/ ft i ft/ ft - /ft 2 l 

ft-/ 0 J ’ 
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where 



W = 



Wit 

W 2 i 



W12] 

0 J' 



is such that WJW'= and 

L=\l Z 2 ] = {BJ& + AZC') . 

The class of controllers (4.18) can be interpreted in terms of an observer/linear 
feedback structure just as in the LQG design problem. The main difference with 
the LQG design is that here no separation principle applies. The observer and 
controller designs are linked. In the above expressions this is seen from the link 
between the controller Riccati equation (4.9) and the filter Riccati equation (4.15) 
via the equations (4.1 1) - (4.14). 



4.5 An l\ Design Approach 

Often in applications the control objective is to keep the tracking error within a 
certain tolerance. An example of this is the regulation of the read/ write head of 
a hard disk drive onto a particular track. Here the control objective is to keep 
the magnitude of the tracking error to within the width of the track. This type of 
control objective leads naturally to an type performance index 

j= IMI00, 

being the infinity norm of the error vector. When the input disturbances w are 
known to be infinity-norm bounded, then J can be reformulated as an l \ index 

* Kjfo II t • 

where Fq is the closed-loop transfer function between the input w to the output 
e. 

Often in the design of controllers, a compromise has to be taken to balance 
objectives for the plant output and the controller output. As discussed in Chapter 3 
and popularized in LQ type controller design, a weighted index is appropriate. 
In the context of l\ design, this becomes either a weighted sum or a weighted 
maximum index given, respectively, as follows 

h = IJ(l.vl+A|«|)|| 0C , 

h= llylloo + *Mloo- 

However, this double penalty approach in an l \ design context does not actually 
achieve the type of effect one would expect it to do, based on experience in the 
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FIGURE 5.1. Limits of performance curve for an infinity norm index for a general system 



LQ design context. To see this, let us consider the solution space of the l\ opti- 
mization problem. The set of all feasible solutions for the control effort amplitude 
and the output amplitude for some particular system forms a convex polygon as 
illustrated in Figure 5.1. 

The boundary of the polygon, termed the limit-of-performance curve, is a plot 
of the best achievable performance for the particular control configuration and is 
constructed using all possible positive weights X in the weighted index function. 
It turns out that this curve consists of only a finite number of linear equations and 
its gradient is monotonically nondecreasing. 

Using a weighted function index results in the solutions of the optimization 
problem remaining unchanged for some range of weights, that is for some val- 
ues of X there are an infinite number of solutions of the optimization prob- 
lem. 

Consider, for example, if the weight is chosen to be any value between the 
gradients of Line 1 and 2 in Figure 5 . 1 , the solution of the 1 1 optimization problem 
remains unchanged since the optimal solutions are at the same vertex. In other 
cases such as when the weight is chosen to be the gradient of Line 3, then the 
resultant l \ optimization problem has an infinite number of solutions along the 
edge of the line. 

In any weighted index approach, the weights are usually chosen, or at least 
fine tuned, by trial and error. Without knowledge of the shape of the solution 
set, a trial and error approach will almost certainly lead to a selection that will 
give a unique solution at a vertex of the solution set. In the event that the weight 
chosen leads to an infinite number of optimal solutions, there is no mechanism 
to select any one of these infinite controllers to practically fulfill the objective of 
compromising between the magnitude of the output signal and magnitude of the 
controller effort. 




Mathematical Preliminaries 

Fact 5.1. Given a linear function f : 
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1. If f has the same value at two distinct points, Y e ” and Z e ", then f 
remains constant along the line Y Z. 

2. If f has different values at Y and Z, then at each point on the open line 
segment YZ, f has a value strictly between its values at Y and Z. 



Fact 5.2. The maximum and minimum values of a linear function f : " — > , 

restricted to a bounded convex polytope A e ", exist and are to be found on the 
boundary of A. 

Fact 5.3. The intersection of any number of convex regions in " is convex. 



Problem Formulation 

Let us consider a single-input, single-output, discrete-time, linear, time-invariant, 
proper system expressed as follows 

(q~ l )yk = iq~ l )uk + ( q~ l )m , (5.1) 

where yk,Uk and wy are the system output, the system input, and the input dis- 
turbance to the system at the £th sample, respectively. The input disturbance Wk 
is assumed to belong to l 0 o with a maximum bound normalized to unity. Here 
(q ~ 1 ) , (q~ l ), and {q ~ 1 ) are polynomials in q~ l given as follows 

{q~ x ) = 1 +a x q~ l + h a np q~ n r , (5.2) 

(q~ l ) =b 0 + b iq ~ l + ■ ■ ■ + b np q- n r, (5.3) 

{q~ x ) = c 0 + ciq~ l + 1- c nc q~ Hc , (5.4) 

with ( q ~ l ) and iq~ x ) assumed to be coprime. Let us consider a stabilizing 
control law for system (5.1) as 



{q l )u k = - {q l )yk, (5.5) 

where 

(q~ l ) = \ + nq~ l + h r„ r q~ nr , (5.6) 

(q~ l ) = so +.si<7 -1 + — \-Sn r q~ nr , (5.7) 

with n r > n p — 1 . Note that when the plant has direct feed through, so is con- 
strained to be zero to avoid an algebraic loop. The system with its controller is 
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FIGURE 5.2. Plant with controller configuration 



shown in Figure 5.2. The closed-loop transfer operators from w k to y k and w k to 
u k are then given as 



yk = 


0T 1 ) ( q - 1 ) 


i 7 TT w k 


=: G y {q ] )w k . 


(5.8) 


Uk = 


Or 1 ) (r X ) + ( q- ] 
- or 1 ) or 1 : 


l ) ( q ~ l ) 

) 

1 7 iT w k 


=: G u (q~ 1 )wk- 


(5.9) 


Let 

Uo = {«,-}; 


Or 1 ) (r x ) + ( q 

ui e , \ui\ < 1, 


l ) ( q~ l ) 

, i e , 


lim Ui =0. 


(5.10) 


We can then wri 

G^- 1 ) ||i 


te 

:= sup G/r 1 )^ 


= sup 


I.v* llytlloo. 


(5.11) 




:= sup G u (q l )w k \ = sup 


l«Jtj =: IlMjtlloo » 


(5.12) 


11 11 1 we£/o 1 1 w€Uq 

Moo=l 

and the following minimization tasks can be defined: 
min Moo = min (4 _1 ) ( '<T 


%• 


(5.13) 




min llmlloo = min 
R.S II 


(q x ) (q 


hi . 

Ill 


(5.14) 


subject to the constraint 

(<T X ) 07 _1 ) + 


or 1 ) or 1 : 


)= 1. 


(5.15) 



A composite minimization task can be defined from the above two tasks. 
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We can, without loss of generality, assign the closed-loop poles to the origin 
with the consequence that the optimal numerator is an infinite impulse response. 
This in turn can be interpreted as a series expansion of the closed-loop operator 
about the origin. 

Alternatively, we can also assign the closed-loop poles to be the stable zeros 
of the polynomial (g -1 ) so that the denominator of the closed loop remains as 
unity. 

Equation (5.15) can be written into a matrix equation as 

M6 = Y, (5.16) 



where 



6 = (r\, . . . , r Hr , so, . . . , s nr )' € 2 " r+1 , 

Y - (-«!, . . . , -a„ p , 0, .... 0)' e n ' +n ? , 

and the matrix M is given as 



f l 0 0 

at 1 o 



0 b\ bo 0 

0 b 2 b\ bo 

■ h ' • • ' • • 



(0 

0 



: ‘••at 10: 

\a np ... at a\ 1 b n 



b 2 bi b 0 

h b 2 h ) 

f (» r +« P )x(2« r +l)_ (517) 



Note that for n r = n p — 1 the polynomials ( q~ l ) and ( q ~ l ) are unique, 
Goodwin and Sin (1984). If n r > n p — 1, however, the polynomials (q~ x ) and 
(i q ~ l ) are no longer unique. In this case write (5.15) in partitioned form as 




(5.18) 



where M is an invertible square matrix of dimension ( n r + n p ), M_ e 
(n r +n p )x(n r +i-n p )^ q e n r +n p an d $ g " r+1_ "p . Note that this is always 
possible under the controllability assumption on (5.1). We can then write 9 as 

9 = M~ l (Y - MO) . (5.19) 

The objective is now to find 0_ that will minimize in some weighted fashion the 
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values of (5.1 1) and (5.12). Let us rewr 



Moo = 


W y 


l 

M~ X (Y -MB) 
9 


Ml 


W u | 


M~ l (Y -M9), 1 






\ 


/here m = n, 


c + : 


1 + n r and f,(0) 



V)x2(n p +1) an( j Wu e 



(5.13) and (5.14) as matrix equations 

-EU-(0.)| =: F ' (0 )> <5.20) 

= £ (5-21) 

> affine in 9 e n r +i-n p and ^ e 

Inp + l) are as f 0 ll 0ws; 





Expressions (5.20) and (5.21) represent the maximum output signal and max- 
imum control signal in terms of the free regulator parameters. Here there is no 
constraint on the values of 9. As long as (5.19) is maintained, the resulting con- 
troller stabilizes system (5.1). With these expressions, we can now seek ways to 
select 9 such that ||y M is minimized in some compromised manner. To do this, 
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first observe that (5.20) and (5.21) can be written in the general affine form for 
fi(0) as 



fi® = J2 a ij^j+ i = 1, ■■■ , 2m; m>n. (5.24) 



Now, it makes sense to use the principles of linear programming (LP) to show 
that the limits of performance curve is defined by finitely many linear equations 
and that its gradient is monotonically nondecreasing. 



Limits-of-Performance Curve 

In the context of the l 0 0 index used here, the limits-of-performance curve is a 
graphical plot of H-yll^ verses 1 1 m 1 1^ . In this subsection, we show that this curve 
is described by a finite number of linear equations with gradients monotonically 
nondecreasing, and we propose a systematic method to construct this curve. 

Let us now define points P G 2 with coordinates (p u , p y ), where p u = F u 09) 
and p y = Fy(6). Let the collection of all feasible coordinate pairs of P be repre- 
sented by the region as shown in Figure 5.3. 




FIGURE 5.3. The region and the required contour line shown in solid line 



Lemma 5.4. Refering to Figure 5.3, consider the region and in particular the 
curve that defines that part of the boundary joining the point P*, where the value 
of p u is minimum, to the point P # , where the value for p y is minimum. (This is 
the solid line shown in Figure 5.3). Then this section of the boundary is described 
by a finite number of linear equations. The vertices of this curve occur at those 
points when n of the 2m equations in (5.24) have intersecting solutions. 
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Proof. The contour concerned can be determined by minimizing a cost function 
of the form 

min(Fy($) + XF U (6)), 0 < X < oo; X e . (5.25) 

0 ' 

For a given X, the solution to this minimization problem will produce a set of 
points (F u (0), Fy(ff)) on the required curve, and thus solving (5.25) for 0 < X < 
oo will achieve all points on this curve. To solve (5.25) for a fixed X, we can 
reformulate the task as 2 2m sets of LP problems. The required solution is the 
minimum value of all the solutions to the 2 2m sets of LP problems. First let us 
define a matrix K of dimension 2 lm x 2 2m with all rows distinct and the value of 
each of its elements either zero or one. In simpler terms, the rows of the matrix K 
generate all possible 2m-bit binary numbers 

0 0 0 ... 0 0 0 

0 0 0 ... 0 0 1 

0 0 0 ... 0 1 0 

0 0 0 ... 0 1 1 

1 1 1 ... 110 



Note that the minimization of (5.25) can be written as 

min^min(^{-l) Ku fi(e) + x J2 subjectto 

(—l) Kti fi(0) > 0; f = 1 .... , 2m; l = 1, . . . , 2 2m J. (5.27) 

Applying Fact 5.3, the region defined in (5.27) for each of the LP problems 
forms a convex region. Applying Facts 5.1 and 5.2, the minimum value of (5.27) 
is at one of the extreme points, which occurs when n of the constraints in (5.27) 
hold with equality. Note that the inequalities do not change the locations of the 
possible extreme points. Hence all the LP problems for 0 < X < oo share 
some of these extreme points, and in total there are not more than 2m C„ = 
(. n(n — 1) ... (n — 2m))/((2m)(2m — 1) ... 1) number of extreme points. 

Since the solution of (5.25), for 0 < X < oo, is the minimum value of all the 
LP problems defined implicitly in (5.27), the solution of (5.25) has to occur at 
one of the 2m C n intersection points. Hence we can conclude that the limits of the 
performance curve is formed by a finite number of straight lines, and the vertices 
occur at those points when n numbers of the equations in (5.25) intersect. □ 

Note that it is not required to solve all the 2 2m LP problems as mentioned above 
to obtain the solution of (5.25). The solution of (5.25) can be solved by evaluating 
the cost at all the 2m C n possible extreme points. 
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Lemma 5.5. The gradients of the curve of the part of the boundary region , as 
described in Lemma 5.4, are monotonically nondecreasing. 

Proof. As mentioned in the previous lemma, this curve is determined by solving 
the minimization problem given in (5.25) for X between zero and infinity. To prove 
that the gradients of the curve are monotonically nondecreasing, we redefine the 
minimization problem (5.25) for a particular A. as described below. But first, let us 
introduce a new variable 0 n+1 . The minimization of (5.25) is equivalent to 

^rnin 9_ n+i , (5.28) 

subject to 

e n+l > (-l)*«/l(0 + • • • + (-1) Ke ”fm(§) + • • • + flrndf 

£ = 1 . . . 2 2m . (5.29) 

Note that each of the inequalities defined in (5.29) is a semiplane that bounds 
a convex region. From Fact 5.3 the region defined by the inequalities forms a 
convex set and its edges are formed by the solution of linear equations. Again 
using Facts 5.1 and 5.2 we see that the solution space of (5.28) is a convex region. 
Consequently, the solutions of (5.25) for a given A lie in a convex region. 

In Lemma 5.4, we have shown that the limits-of-performance curve is formed 
by the solution of a finite number of linear equations. Therefore, minimization 
of (5.25) has either a single solution or an infinite number of solutions. Here we 
have shown that in the case where there are an infinite number of solutions, the 
solutions lie in a convex region. We conclude that the gradients of the limits-of- 
performance curve are strictly monotonically nondecreasing. □ 

From the above two lemmas, we conclude that the limits-of-performance curve 
are formed by the solution of a set of linear equations with gradients monotoni- 
cally nondecreasing. Let us now present a systematic method for constructing this 
curve. The steps are given as follows: 

Algorithm. 

StepO. Find all the intersection points of the 2m equations (— 1 ) Ka fi(g) > 0; 
i — ] ,2m; l = l,...,2 2m and calculate the coordinates 

(Hylloo > 1 1 m I loo ) at all the points. This involves solving 2m C n sets of n 
simultaneous equations and substituting the solution back to (5.20) and 
(5.21) to determine the coordinates of (llylloo , ||«Hoo)- Determine the 
point P* where 1 1 >’ 1 1 ^ is minimum. Label these two pairs of points as 
P 0 and Pi respectively. Let i = 0 and j — 1. Go to Step 1. 

Step 1. If j < i, go to Step 4, otherwise determine the line that joins P, to Pi+\. 
Find from among the rest of the intersection points the one that has the 
smallest displacement from this line. Label this point as /min- If the min- 
imum displacement is smaller than zero, go to Step 2, otherwise go to 
Step 3. 
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Step 2. For k = j to i + 1 in decrements of 1, set Pk+i = Pk and P,-+ 1 = P m 'm- 
Set j = j + 1 . Go to Step 1 . 

Step 3. i = i + 1. Go to Step 1. 

Step 4. The contour of the limits-of-performance linking the minimum achiev- 
able || m H oo to the minimum achievable ||}j| oc is now defined by the 
straight lines joining points Pq to P\ , Pi to P2, , Pj-i to Pj. 

Remark. Due to the nature of the problem, one is required to solve 2 m C„ sets 
of n simultaneous equations. Therefore this method may not be computationally 
feasible for plants of very high orders (n p > 20) or plants that use a relatively 
high order regulator. Table 5.1 shows the regulator order for a plant of varying 
order. Here F is the number of flops required to solve a set of n simultaneous 
equations, and T_F is the total number of MFLOPS required to solve 2m C n sets 
of n simultaneous equations. 

We have not explored the possibility of using efficient combinatorial algorithms 
such as the so-called branch and bound method to solve this problem. Such tech- 
niques could prove useful, especially for large-scale problems, and the interested 
reader could consult Nemhauser and Wolsey (1988). 



n p 


m 


» 


» 


2m C n 


F/Flops 


T_F/MFLOPS 


2 


li 


12 


10 


1.96 x 10 6 


2500 


4.9 x 10 3 


4 


12 


13 


9 


3.12 x 10 6 


1840 


5.8 x 10 3 


5 


13 


14 


9 


6.9 x 10 6 


1840 


13 x 10 3 


8 


15 


16 


8 


10.5 x 10 6 


1350 


14 x 10 3 


10 


16 


17 


7 


5.3 x 10 6 


950 


5.2 x 10 3 


15 


20 


21 


6 


5.25 x 10 6 


620 


3.2 x 10 3 


20 


25 


26 


6 


20.3 x 10 6 


620 


13 x 10 3 



TABLE 5.1. System and regulator order and estimated computation effort 



Example. In this section, we will present the limits-of-performance curve for 
a second-order single-input, single-output, linear, time-invariant system. It has a 
pure unit delay, a nonminimum phase zero at z = 2.0 and two poles at z = 1.2, 
and z = 0.7. The system is given as 

(l - 1.2<7 _1 ^ (l - 0.5q~ l ^j y k = q~ l (l - 2 q~^ u k + w k . (5.30) 

Let us consider regulators with a fifth- and eighth-order structure, respectively. 
The limits-of-performance curve for the eighth-order structure is shown in Fig- 
ure 5.4. For the fifth-order regulator, the lines that make up the limits of perfor- 
mance curve are in effect the lowest three segments having gradients of — 1 .997, 
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— 1.189 and —0.476 respectively. If X happens to belong to this set, then the solu- 
tions that satisfy the optimization problem will be infinite in number. If, however, 
X happens to be between these values, then the solution for the optimal control 
will remain unchanged. 

From this example, it is clear that the weighted-sum method is not a suitable 
method for solving the optimization problem. For most of the weights chosen, the 
solutions remain the same. In the unlikely event where some particular weights are 
chosen, there are an infinite number of solutions. Given the limits-of-performance 
curve, one can now decide the compromise one has to make and select the appro- 
priate operating point. For example, assuming the bound of the noise is normal- 
ized to unity and if the control effort is constrained to 3.5 units, the worst output 
signal will then be 6. 1 units (shown as point A in Figure 5.4). In the case when the 
constraint on the output signal can be relaxed a little, we can choose to operate at 
a point such that the maximum output signal is less than 6.65 units with maximum 
control effort less than 2.5 units (shown as point B in Figure 5.4). In the case of 
unconstrained control effort, the best we can achieve for the output signal will be 
5.7 units and the worst control effort is 4.4 units. 




FIGURE 5.4. Limits-of-performance curve 



Main Points of Section 

In this section, the limitations of using the weighted-sum method or the weighted- 
maximum method for solving l\ optimization problems are illustrated. A limits- 
of-performance method to overcome this problem is described and the procedure 
to obtain this curve is illustrated. The advantage of this approach is that now 
we can do away with the trial and error method of selecting an appropriate cost 
function. Also the performance of the regulator structure can be observed and 
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altered if necessary. Another advantage of this method is that with the knowledge 
of the maximum bound of the input disturbance, one can select the operating point 
so as to minimize one or more of the signals while keeping the other within its 
constraints. 



4.6 Notes and References 

This chapter aims to collect together in one place some contemporary controller 
design techniques. The stress is not on proving theorems, but rather on presenting 
the practical techniques to allow practicing engineers to quickly implement such 
controllers for their plants. 

We begin with a discussion on performance specifications. These specifications 
can be found in almost every controller design based paper. However a good start 
would be books such as Boyd and Barratt (1991) and Vidyasagar (1985). 

Linear quadratic control has been well explored. Good references are Ander- 
son and Moore (1989) and Kwakernaak and Sivan (1972). The need for LQG/LTR 
was first raised in Doyle (1974) indicating a poor stability margin as a result of 
using state estimate feedback rather than state feedback. Subsequently stability 
margin recovery (LTR) methods were developed by researchers such as Doyle 
and Stein (1979), Zhang and Freudenberg (1987), Moore and Xia (1987), Moore, 
Gangsaas and Blight (1982), Lehtomaki, Sandell and Athans (1981), Stein and 
Athans (1987). The material of Section 4.3 is obtained from Moore and Tay 
(1989c), which is in many ways an advance on the previous works. It introduces 
the concept of sensitivity recovery, and this is done via a properly selected Q. 

For l\ optimal design, the problem was first formulated by Vidyasagar (1986). 
Subsequently the work to solve the optimization problem for various situations 
were undertaken by Dahleh and Pearson (1987), Dahleh and Pearson (1988), 
Vidyasagar (1991) and their coworkers. The key approach in all these works is 
to formulate the desired closed-loop transfer function in terms of Q and then 
subsequently to convert the optimization into a linear programming problem by 
means of a dual formulation. The technique described in this chapter is based 
on the work by Teo and Tay (1995) and takes a polynomial setting. It takes the 
i i solution a step further towards a practical design by proposing a technique to 
construct the entire limits-of-performance curve for all possible control energy 
penalties in a weighted sum performance index. 




CHAPTER 5 



Iterated and Nested (Q, S) 
Design 

5.1 Introduction 

In Chapter 4, we have presented controller design strategies to achieve various 
control performance objectives. For some of these methods the Q parameteriza- 
tion of Chapter 2 proved helpful. The assumption behind all the strategies is that 
the plant model is known. The controller is designed to reject in some optimal 
fashion certain classes of disturbances applied to this nominal model. There is 
no explicit provision to cope with plant perturbations or plant uncertainties in the 
designs. Of course it turns out that some of the designs are also robust to cer- 
tain classes of plant perturbation or uncertainty. There can be a trade off between 
performance and robustness, but robustness to prescribed plant variations or un- 
certainties is not included explicitly in the optimization criteria, so that this trade 
off may not be straight forward. 

In this chapter, we take a different approach to controller design from that of us- 
ing the standard optimization techniques presented in Chapter 4. We will examine 
a controller design strategy that will take account of unmodeled dynamics in the 
nominal model of the plant. Here we view the controller K(Q) as a nested two 
controller structure, consisting of the nominal controller K and an augmented 
controller Q, in the notation of Chapter 2, with K = K(Q)\q =0 = K (0). In 
the same vein, we also view a plant G(S) as consisting of two parts; a nom- 
inal part G and an augmentation we call S in the notation of Chapter 3, with 
G = G{S) | s _o = G(0). The augmentation S is deemed to contain dynam- 
ics that are not modeled in the nominal part G of the actual plant. In fact, 
we have shown in Chapter 3 that the nominal model can be viewed as a sim- 
plified model of the actual plant obtained from, say, an initial identification 
procedure. The augmentation S then turns out to be a frequency-shaped de- 
viation of the nominal model from the actual plant, with the frequency shap- 
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ing emphasizing the cross-over frequencies of the nominal system and of the 
actual system. 

The strategy adopted here is to first design the nominal controller K to opti- 
mally control the nominal plant G. The initially unmodeled dynamics represented 
by S is then identified leading to an estimate of S. This process can be carried 
out either on-line or off-line from measurements, as discussed in Chapter 3 and 
elaborated subsequently. Once an S is identified, the next stage is to design an 
augmented controller Q to optimally control S according to some performance 
measure, related to, but not necessarily identical to that of the initial controller 
design. This approach exploits the robust stabilization results in Chapter 3 where 
it is shown that when K stabilizes G, then K (Q) stabilizes G(S) if and only if Q 
stabilizes S. 

In this chapter, we also take the work of Chapter 3 a step further by show- 
ing that from a performance point of view, the approach to design the nom- 
inal controller K for the nominal plant G, and the augmented controller Q 
for the derived plant S can be complementary. In fact there need be little 
danger of the augmented controller Q interacting with the nominal controller 
K in an adverse manner so that the original control object is compromised. 
In particular, we demonstrate how to achieve the complementary behavior for 
the familiar design methods of pole-placement, linear quadratic control and 
tfoo control. We show that for these three techniques, designing the Q to 
control S, using the same criterion as that used in the design of the nomi- 
nal controller K for G, ultimately assists in the achievement of the original 
design goal. 

We then move on to extend the results of Chapter 3 in another direction. We 
show that in general, there need not be a limit to a nested two-controller struc- 
ture. In fact, any plant can be represented in an m recursive fractional form via a 
continued fraction expansion. The results in Chapter 3 correspond to the spe- 
cial case where tn — 2. Similarly, instead of the two-controller nested struc- 
ture consisting of K and Q, we can generalize the two-controller structure to 
an m-controller nested structure for a plant represented in an m recursive frac- 
tional form. The stability results for the two-controller structure are then gen- 
eralized to this multiple controller structure. With this generalization, we pro- 
pose a multistage or an iterative controller design approach as a natural ex- 
tension to the two-stage controller design approach. In some cases, a high or- 
der original plant may be decomposed to form a sequence of relatively sim- 
ple plant models which allow successive approximation of the plant. The task 
of designing a complex controller for the high order plant can then be broken 
down into a sequence of lower order controller designs for a sequence of lower 
order models. 

The iterated or multistage design methods of this chapter are focussed on a 
sequence of off-line controller improvements based on on-line identification, but 
the results lead naturally to on-line adaptive control as developed in the following 
two chapters. 
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The results of Chapters 2 and 3 allow us to proceed with a controller design as a 
two-stage process. 

The first stage is the design of a stabilizing nominal controller K for a nominal 
plant G, but also stabilizing the actual plant G(S). In this section, we will think of 
this stage as an off-line design where we will do our best to obtain a controller for 
the plant based on the a priori knowledge of the actual plant, represented by the 
nominal model of the plant. Thus if there are no unmodeled dynamics, then this 
off-line designed controller should be the optimal controller for the actual plant, 
but if there are plant uncertainties the controller should be robust to these; maybe 
not achieving good performance, but at least ensuring stability. 

We will view the second stage as an enhancement stage. Here we will utilize 
on-line measurements to identify the unmodeled dynamics in the original plant, 
represented by S. The matrix transfer function S is a frequency- shaped deviation 
of the nominal plant model from the actual plant. The augmented controller Q is 
then designed based on the identified S. The design method for Q is appropriately 
selected to ensure that there is no conflict of the original controller design goal. 
This can be interpreted as stating that after the implementation of Q, the imple- 
mented controller K(Q) actually solves the design problem for the plant G(S). 
Figure 2.1 illustrates the iterative- Q design process. 

It is clear that the overall ^-enhancement procedure can be iterated. Indeed the 
identification of S may have been incomplete, or due to the new updated design 
deficiencies in the model may have been accentuated. Both may necessitate a 
repeating of the procedure. 



Identification of Unmodeled Dynamics 

Let us consider the problem of identifying S given that a particular controller 
with a Q augmentation has been implemented, as given in Figure 2.1. The matrix 
transfer function J and the operator Q form the feedback controller K (Q) for 
the plant G(S). For the purpose of the iterative scheme, the plant G(S) and the 
matrix transfer function J are viewed as a single block W , such as is depicted in 
Figure 2.1. The signals r,s,w\,u,v, and y are assumed to be measurable. The re- 
lationship between r, s, and wi,W2 can be written from the key operator equation 
(3.4.22) derived in Lemma 3.4.3 specialized to the representation of Figure 2.1 
(see also (3.6.3)), and is given by 

r = Ss + M(S)w 2 + N(S)wi. (2.1) 

This is a linear equation in S and can be reformulated into a linear regression 
equation for a standard identification algorithm. 

Besides the relationship expressed in (2.1) we also have the control relation- 
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Output disturbance 




FIGURE 2.1. An iterative- Q design 



ship: 



s = Qr- (2.2) 

Recall also from (2.12) and (2.13) that, in the absence of w, 

r = My — Nu, s = Vu - Uy. (2.3) 

Notice that in the first stage of the redesign process Q — 0 applies, as the only 
controller implemented is K — K (0). In this situation (2.1) suffices for identi- 
fication purposes. Unfortunately, due to the obvious feedback interconnection of 
(2.1), (2.2) it is unclear how to obtain an unbiased estimate for S, at least without 
adding excitation signals to s. Moreover, even without the feedback interconnec- 
tion, due to the fact that in (2.1) the transfer functions between s,w\,W2 and r 
are not independently parameterized, it might be thought difficult to obtain an un- 
biased estimate of S. The problem is overcome without adding extra signals as 
follows. 

Intuitively, in view of the closed loop (2.1), (2.2) it is clear that we can express 
both r and s in an affine manner in terms of the closed-loop transfer function of the 
feedback system (Q, .S'), and the external signals w\ and u>2- If Q is known, we 
could then deduce S from knowledge of (Q. .S'). This is achieved in the following 
development, see also Figure 2.2. 

In order to fix ideas, we assume here that the signal W2 is an unmeasurable 
disturbance while w\ is measurable. Any other combination can be treated in a 
similar fashion. Moreover, let us assume that w\ and W2 are uncorrelated signals, 
as defined in Section 3.2. 
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FIGURE 2.2. Closed-loop identification 

Let us clarify the assumptions on G(S), K (Q) for the following. We assume 
that (G = G(0), K = K (0)) form a stabilizing pair, and that (G( 0), K(Q )) and 
(G(S), K(Q )) are stabilizing. Thus we have assumed both that Q is stable and 
that Q stabilizes S. 

With the above stability assumptions in mind, let us introduce coprime factor- 
izations, 

G = G( 0) = M~ l N = NM~\ 

. i (2-4) 

K = K( 0) = V~ l U = VU~\ 

and 

K(Q) = U Q V~ 1 = Vq'Uq, (2.5) 

where 

U q = U + MQ, Vq = V + NQ, 

Uq = U + QM, V q = V+QN. 

Also, 

G(5) = (M + SU)-\N + SV) = (N + U S)(M + VS) ', (2.7) 

and we introduce coprime factorizations for G(S) parameterized in terms of S or 
S as 

G(S) = G q (S) = Mq(S)~ 1 Nq(S) = N Q (S)M Q (Sr\ ( 2 . 8 ) 



Nq(S) = N + VqS, 
Nq(S) = N + SVq, 



where 



Mq(S) = M + UqS, 
Mq(S) = M + SUq, 



(2.9) 
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(see also equation (3.4.3)). Also we have the double Bezout identity 




Let us introduce the auxiliary variables a, fi given from 




The importance of the signals a, P for identification purposes can be inferred by 
expressing a, P in terms of the external signals driving the system in Figure 2.1. 
With reference to Figure 2.1 it is clear that, since G(S) = Gq(S), 

H r / -*(G)i -1 m 

L-gqgs) / J W 

where simple manipulations show 

I -K{Q)V X 

- Gq(S ) I \ 

MqCs ) u Q ] \v q o 1 

Nq(S) v e J L 0 Mq(S) \ 

MqCs)] r - - i r° -m q Cs)Uq + UqMqCs) 1 

We(5)JL e [o -Nq{S)Uq + v q Mq{S)\ 

».)*[: :]■ 

The last equality follows from the double Bezout equation (2.10) and the defini- 
tion of Mq(S ) and Nq(S). It follows then that using the definitions in (2.11), and 
the Bezout identity, we get 

[:]-K i](["S]['« »*]* 

■([']['* »•]*[: -».])[:] 

sv Q m 2 (S)T|W] 

Vq 0 J [w 2 \ ’ 

from which we deduce that a is independent of wj, and 

P = Soc + Mq(S)w 2 . (2.12) 
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This has the interpretation that we can obtain an unbiased estimate of S from 
measurements of f and a via, for example, output error identification schemes. 
For a treatment of open-loop identification methods, we refer the reader to 
Ljung (1987). An in depth treatise of the ideas presented above on closed- 
loop identification can be found in Lee (1994), Schrama (1992b) and Hansen 
(1989). 

The above results are summarized as the following lemma. 

Lemma 2.1 (Unbiased closed-loop identification). Refer to the Figure 2.1. As- 
sume that the controller K(Q ) stabilizes both the plant G(S ) and the nomi- 
nal plant G( 0). Let K( 0) = UV~ X = V~ l U, G( 0) = M~ l N = NM~ l , 
K(Q ) = UqVq 1 = V Q U~\ as in (2.4). Define the measurable signals a, ft via 
(2.11). Then as depicted in Figure 2.2 

P = Sot + (M + SUq)w 2 , a- Vquu. (2.13) 

Provided the external signals w\ and u>2 are uncorrelated, it follows that S can 
be identified in an unbiased manner via correlation analysis of ft and a. 

Now S and S are related by (2.8), in that 

(N + V S)(M + US r 1 = (M + SUq)-\N + SV Q ), 

where Uq = U + QM, Vq = V + QN. Hence we can deduce that 

S=(I + SQy l S = S(I+QSr\ (2.14) 

or 

S = S(I - QSy 1 = (I - SQ)~ l S. (2.15) 

These expressions allow us to infer S from S and vice versa. Reinterpreting the 
result of Lemma 2.1 in the light of the expressions (2.14) and (2.15) we come to 
the rather obvious statement that we are able to identify the closed-loop system 
(Q, S) from the data in an unbiased manner. This is depicted in Figure 2.2. 



Iterated Control-identification Principle 

Since we start with the assumption that K(Q ) stabilizes G(S), once we obtain 
an estimate for S, we are guaranteed that the present controller K (Q) will also 
stabilize Gq(S ) since Gq(S ) = G(.S'J under (2.14), (2.15)). We are now in a 
position to update the controller Q so as to obtain an improved closed loop. Indeed 
having identified S we can find an associated estimated S via equation (2.14). For 
this estimated S we can now design a new Q and the whole procedure can be 
repeated if desired. 
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One step iteration 




FIGURE 2.3. Iterated- 2 design 

We stress that it is important to base the identification on the S representation 
rather that the S representation. This is because unbiased S estimates can be ob- 
tained from standard identification algorithms. This is not the case for S estimates, 
at least without the addition of excitation signals in the Q, S loop. 

Because we now want to redesign Q so as to stabilize the newly found S while 
achieving some design goal, and because of the relationship between S, S and 
Q , we have that the stabilizing pair (Q\, .S'), where Q \ denotes the new Q, is 
identical to the stabilizing pair (S, Q) where Q = Q\ — Q. We may therefore 
proceed with designing a stabilizing controller for S and augment the existing Q 
by this new controller to find the actual controller to be implemented. 

The process is summarized in Figure 2.3 and detailed in the next algorithm. 
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Algorithm. 

StepO. •Initialize Go = NM~ l — M~ l N , 

K 0 = UV~ l = V~ l U, 

Qo = 2o = 0, So = So = 0, 

1 = 0. 

Step 1. • V e = V + QiN, U e = U + Q e M , 

M t: = M + S t U e , N t = N + S t Ve, 

K t = G t = 

at = Vtv — Uty = V t wi, fit = —Nv + My. 

• Identify S from fa = Sat + (M + SUt)u >2 using an output error 
identification method. 

• Update control by designing (S, Q). 

Step 2. • Output 5f+i = S, 

Qe + 1 = 2^ + 2- 

Step 3. • Either let £ + 1 be l, go to Step 1. 

• Or stop if control objective is achieved. 

The key idea in an iterated control design is thus to first stabilize the plant 
with some nominal robust controller, presumably not achieving high performance. 
Consequently, with the stabilizing controller in the loop we reidentify the system, 
or better the 5-factor representation. Then we redesign the controller, or better the 
2 parameter, to obtain improved performance. This may be iterated. The reason 
why iterations are necessary stems from the fact that no one iterate is capable of 
representing the plant accurately. The success of this method hinges on our ability 
to use low order approximations for 5 as is exemplified in Chapter 3. Even then, 
for practical implementation, controller order reduction may be required, as each 
iteration augments the controller order. 

We now discuss how the control design for some specific control objectives can 
proceed in this iterated framework. 

Iterated Pole-placement Strategy 

We now present a result to show the rationale behind designing Q using a pole- 
placement algorithm. Let us examine the relationship among the closed-loop sys- 
tems formed by the pair (G(5), K (Q)) representing the actual closed loop, the 
pair (G, K ) representing the nominal or design control loop and the pair ( Q , 5). 
We show that the eigenvalues of the closed-loop system formed by (G(5), K (Q)) 
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consists of the set of eigenvalues of the actual closed-loop system represented by 
(G, K) together with the set of eigenvalues of the closed-loop system (Q, S ). We 
summarize the result in the following theorem. 

Theorem 2.2. Let (G, K) be a stabilizing nominal plant-controller pair in that 
(2.3.5), (2.3.6) holds. Let G(S ) be the class of plants parameterized by S as in 
(2.7) with G = G(0), and let K(Q) be the class of controllers parameterized by 
Q as in (2.4) with K = K (0). Under these conditions, the set of closed-loop poles 
of the pair (G(S), K ( Q )) is generically the union of the set of poles of the pairs 
(G, K) and (Q, S). 

Proof. First recall equation (3.4.18) from Chapter 3, 




from which the result follows, since it turns out that the coefficient matrices in 
the second term introduce no additional dynamics to the dynamics of the first 
term. Notice that any pole/zero cancellations necessarily involve stable pole/zero 
cancellation by construction, as M, U, N, V, M, U, N, V all represent stable 
rational transfer functions, as well as (Q, S). □ 

This theorem indicates that if the nominal controller K for the nominal plant G 
is designed based on a pole-placement technique, then the additional poles arising 
in the case of unmodeled dynamics can be assigned to appropriate locations using 
the ‘controller’ Q. There is no conflict with design objectives in each stage of the 
design. 

Other important interpretations are obtained by considering the closed-loop ar- 
rangement (G(S), K (0)) as the plant to be controlled by Q. The closed-loop be- 
havior, according to Theorem 2.2, is characterized by the actual desired design 
behavior (G, K) together with that of (S, 0). In other words, S models the dif- 
ference between the desired and the actual closed loop. Therefore, Theorem 2.2 
implies that in order to achieve a desired control objective such as a certain con- 
trol bandwidth, one should design the nominal controller K so as to achieve this 
design goal on the nominal plant model G and iterate the same design goal for the 
control loop (Q, S). 

The limitation with the above methodology is that in the presence of significant 
model errors, our control objective must be a cautious one. Otherwise, achiev- 
ing high performance on a nominal design could lead to an unstable closed-loop 
response for (G(S), K( 0)), or equivalently, an unstable S. Indeed, the above the- 
ory allows for this since an unstable S can be stabilized by an appropriate Q. 
Nominally however, it makes sense to proceed with a cautious design objective, 
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and iterate to an improved model for the plant keeping the same control objec- 
tive. When the nominal model response and the actual closed-loop response are 
in close agreement, we can then envisage changing the control objective, that is, 
changing the desired closed-loop poles. The whole process can then be started 
over, until such time that we are satisfied with the closed-loop response. 

From the previous discussion on iterated design and Theorem 2.2, it follows 
that at any stage of an iterated pole placement design, the closed-loop poles of 
the system are the poles of (G, K) and (S n , Q n ) = (S, Qi + Qi + • • • + Q n )- 
Therefore, at no stage in the iterative design process is there a conflict in the 
design process, even when we work with S, Q variables. 

Example. Let G be a second order plant with poles at 2.348 4 ± j 1 .628 2 and K 
be a pole placement controller that assigns the closed-loop poles to the locations 
(0.8, 0.8, 0.7, 0.7). The parameters of the various transfer functions are given in 
Table 2.1. Let us work with a first order S and let Q be the controller that assigns 
the closed-loop poles of (Q, S ) to (0.75, 0.75). 

We now construct G(S) and K(Q). Notice that the closed-loop transfer func- 
tion G(S)(1 — K(Q)G(S))~ l as given in Table 2.1 has identical poles to those 
G(1 — KG) -1 and 5(1 — QS)~ l , as predicted by Theorem 2.2. 

Iterated Linear Quadratic Design Strategy 

We first present a result which explains the rationale behind the design of con- 
troller K(Q) using the linear quadratic technique. We refer to Figure 2.1. All 
external disturbances are assumed to be zero, i.e. w\ = W 2 = 0. (In this case, 
u = v in Figure 2.1). 

Theorem 2.3. With (G, K) a stabilizing plant-controller pair, consider a plant 
G(S) with a controller K(Q) applied for some Q, as in Figure 2.1, see also 
factorizations (2.4), (2.7). Consider also a linear quadratic index, penalizing the 
controller internal signals r and s (being the inputs and outputs of Q, respec- 
tively), as 



Jlq = lim y^fr-rj + s- Rst ) . 



(2.16) 



where R is a symmetric positive definite weighting matrix. Then this index can be 
expressed in terms of a frequency-shaped penalty on the plant outputs and inputs, 
y, u as 



Jlq 



lim 

k — >-oo 






(M*M + U*RU) 
—(N*M + V*RU) 



~(M*N + U*RV) 
(N*N + V*RV) 



(2.17) 



where * denotes conjugate transpose. 




138 Chapter 5. Iterated and Nested (Q, S) Design 



Transfer functions 


Poles 


r 1. 389 5z _1 -2.879 3z“ 2 

l-4.6969z- 1 +8.1662z" 2 


2.348 4 ± j 1.628 2 


K _ ^.loosz-'+w.ogogz- 2 
1+1. 696 -6.692 8z~ 2 


-3.571 1, 1.8742 


G(1 — KG)~ l 

1.389 Sz^-O^Mz^-M.lSS 5z -3 +19.2703z -4 


0.8, 0.8, 0.7, 0.7 


l-sz^+s^z-z-i.esz-s+o.suesz- 4 


5 = g§j| 


0.9 


/■» -0.022 5z"‘ 

^ 1— 0.6z -1 


0.6 


5d qst x - 


0.75, 0.75 


r , cn 2.389 5z -1 —2.433z -2 — 4.101 4 z" 3 

K J 1 5.596 9z 1 | 5.292 6z 2 i 11.74! 3z“ 3 


3.2946 ± y 0.988 4, 
-0.9924 


K(( ~f \ _ -7.123 3z _1 +23.457z _2 —l 1.638 3z -3 
1—1.096 9z _1 —7.742 2z -2 +4.804z -3 


-3.578 8, 1.873 3, 
0.6087 


G(S)(l-tf (Q)G(S))" 1 

2.389 5z - M-0.188 1 z - 2 -25.087 8z" 3 

+24.087 8z _4 +21.826 3z“ 5 -16.735 63z“ 6 
1— 4.5z -1 +8.4325z -2 — 8.4225z -3 
+4.729 2z“ 4 - 1.415 4z“ 5 +0. 176 4z" 6 


0.8, 0.8, 0.75, 
0.75, 0.7, 0.7 



TABLE 2.1. Transfer functions 



Proof. We have that with w\ = W2 = 0: 




(2.18) 



Substituting this expression into (2.16), the result follows. 



□ 



Example. We use the nominal plant G given in Table 2.1. An LQG controller 
penalizing the index 



Ji = ^lim J^(yf + Xu b> x = 0 I > 



is used to design the nominal controller K. Similarly, the transfer function S of 
Table 2.1 is used to design Q using the index 

k 

J 2 = ^lim y^(r? + Xsf), 



X = 0 . 1 . 
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FIGURE 2.5. Frequency shaping for 
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Normalized frequency 



FIGURE 2.6. Closed-loop frequency responses 



Figure 2.4 and Figure 2.5 show the magnitude/phase plots of ( M*M + U*X 2 U) 
and (N*N + V*X 2 V) which are, respectively, the frequency shaped penalties for 
y and u as a result of using the index J2. The frequency shapings exhibit a higher 
penalty at the low frequency range for y, and conversely for u. Figure 2.6 shows 
the frequency responses for the closed-loop transfer functions. 

Theorem 2.3 tells us that performing an LQ design based on the penalty of r and 
s is equivalent to performing an LQ design based on a frequency shaped penalty 
of y and u as in (2.17). We recall that M, N, U, V reflect the closed-loop poles of 
the pair (G, K ). Thus in this case, we can interpret the frequency shaping given 
in (2. 17) as an emphasis on y, u in the pass band of the nominal design, hence in 
the frequency bands of interest. It is therefore a meaningful index to minimize. 



Iterated Design Strategy 

In order to fix ideas, we discuss here an iterative H 0 Q design in the case where the 
performance variable is simply e = [ “ ]. The generalized plant description then 
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takes the form, referring to Figure 2.1, with G = G(S), 




(2.19) 



The standard H^o problem for (2.19) is one of designing a controller K as to 
minimize the H 0 0 norm of the transfer function matrix from the disturbances w = 
[ ] to the performance variable e = [ " ]. 

Here we would like to approach the H 0 Q optimization problem in an iterated 
fashion. Let us first solve the H 0 Q problem for the nominal plant, and next perform 
the Hoo optimization problem for the unmodeled dynamics. The first subproblem 
is solved in terms of K and G, and the second is solved in terms of S and Q. 

It remains to be seen how this suboptimal H 0 0 control design method for the 
actual plant G(S ) is related to the complete H 0 0 control design problem. Working 
with our standard notation, let G = G{S), K — K(Q). Assume that K solves 
the Hoo problem for the nominal plant model P, in that K minimizes the H^o 
norm of the transfer function matrix (P, K ) from the disturbances w — \Z\\ 
to the performance variable e — [ “ ] in the nominal system model. This transfer 
function is given by: 



Here P is given by 



>-[: HI 

-[-« :] 

.It: :] i 



K\I-GK\ G / 



The difference between the actual transfer function linking disturbances and per- 
formance variable and its nominal version is given by 






Using the expression derived in Chapter 3, see (3.4.18) Theorem 3.4.2, this may 
be re-expressed as 



(P,K)~ ( P,K) = 



~M U] I" / -Q~\ 1 \V Ul 
N V -S I N Mj 
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Clearly, this can be reinterpreted as a frequency weighted left fractional represen- 
tation for the system: 



/] 

S 



■t :]•[;] 

■[-. ".T-E :] 



QV-SQT l \s i\ 



The above expressions lead to the following theorem. 



Theorem 2.4. Consider the block diagram of Figure 2.1. Let the model be as 
defined in (2.19) and let the controller be K — K(Q ) of (2.5). Under these 
conditions we have 

= U \ < s ' 2 >D a\ 

It follows that the design of K for the plant P can be approximated by 
solving two Hoo design problems. The first H 0 0 design is to find an optimal con- 
troller K for the nominal plant P, the next H 0 0 design is a frequency weighted 
Hoo design of the <2 -factor on the plant S. Clearly, this leads to a suboptimal H 0 0 
design for the plant, as indeed: 



*f\\ (A^)lloo 

. II [m t/1 Av u~\ 

< mm || {P, A'JIloo + min (S, g) _ 

K 71100 Q | [at yj V N M\ 

Even so, the additional cost may well be acceptable since the iterated design pro- 
cess may at every stage be much less complex than solving the overall problem at 
once. The following example illustrates the principle. 

Example. Consider a true plant G as 



0.997 6 0.0464 -0.0002 

-0.093 8 0.8575 -0.0162 

-0.003 6 -0.003 6 0.813 8 

-0.008 8 -0.008 8 -0.1600 

-0.1242 -0.124 2 0.1024 

-0.0351 -0.0351 -0.0774 



0.006 6 0.0007 -0.0002 

0.217 4 0.0376 -0.0106 

-0.3731 0.0639 -0.0086 

0.3598 0.1964 -0.0395 

-1.425 5 0.7347 0.098 8 

0.7071 -0.8844 0.3290 



-1.5000 -5.0000 -3.000 0 2.5000 -2.000 0 0 | 0 



(2.23) 
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FIGURE 2.7. Modeling error || G — G|| 
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FIGURE 2.8. Magnitude and phase plots of (P, K), ( P,K ) 
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FIGURE 2.9. Magnitude and phase plots of ( P , K (Q)) 
and a nominal generalized plant P — j p'* j with P 22 = G as 



(2.24) 



The resulting modeling error G — G is depicted in Figure 2.7. An optimal H c 
controller is found to be 



0.9524 -0.053 8 -0.010 5 0.0141 

0.006 1 0.6539 -0.1889 -0.0587 

0 0.1900 1.056 1 0.097 8 

0 0 0.1549 0.7222 



-0.023 2 
-0.1476 
0.1101 
0.0495 



0.203 5 -0.0150 



(2.25) 



It turns out that K stabilizes both G and G. The magnitude plots of (P, K), 
(P, K ) are depicted in Figure 2.8. The Woo -norm of the closed-loop transfer 
function (P , K ) is well above that for (P. K ). although in most of the fre- 
quency spectrum || (P , K ) || is virtually identical to || (P, AT) || , see Figure 2.8. 
The difference at low frequencies is due to unmodeled dynamics. 
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To design the additional controller Q , we choose V,U, N and M based on G 
and K. Thereby the frequency-shaped modeling error S can be generated. But, 
we only use its second order balanced-truncation model to form the generalized 
error model S via (2.21). Then an optimal H 0 0 controller Q for S is designed. 

The controller K(Q) = (U + MQ)(V + NQ)~ l evidently stabilizes G. Not 
unexpectedly, || (P.K (Q)) || is dramatically reduced at low frequencies relative 
to | (P, K) || , see Figure 2.9. 



Main Points of Section 

Off-line controller design using any of the available methods may not lead to 
controllers that perform well on an actual plant. Reidentification can be used to 
achieve improved controllers. The methods proposed here do not reidentify the 
plant itself, but rather identify a version of the difference between the plant and 
its model used for an initial design, denoted S. A controller Q is now designed 
for S , working with performance objectives which do not conflict with those of 
the initial design, but rather support these objectives. The controller Q is then a 
‘plug-in’ controller for augmentation of the initial feedback control closed-loop. 

We observe that, in general, it is not possible to identify S accurately from 
closed-loop data. Rather, we identify S = (I — SQ)~ l S, which is in one-to- 
one correspondence with S, given Q. In the case of pole placement design, we 
could proceed with an iterated control-identification design using S, Q. However, 
for /fyo and LQ control design, it is important to recover S and then design Q 
accordingly. 



5.3 Nested (Q, S) Design 

In this section, we redevelop the ideas of the previous section. The central idea 
is to realize that given a plant-controller pair ( G(S ), K (Q)), the design of a con- 
troller may be viewed as a design involving the pair (Q, S ). Clearly it is then 
possible to view this as a problem of the form (Q(Qi), 5'(5j )), etc. This heuristic 
is made more precise in the sequel. The presentation here builds on the results 
established in Section 5.2. 

First we justify the nested (Q, S) identification-control cycle in a heuristic 
manner. Then we develop a number of representation results to establish that the 
nested (Q, S ) procedure is capable of achieving arbitrary control objectives. 



Heuristics 

To fix the ideas, let G represent the plant and denote G = Go as an initial 
model. Let Ko denote an initial stabilizing controller, stabilizing both the nominal 
plant model Go as well as the plant G. Consider factorizations Go = NoMf 1 = 
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FIGURE 3.1. Step 1 in nested design 



M 0 1 No and Kq = UqV {) 1 = V () 1 Uo satisfying the double Bezout equation 

| ~M 0 t/ 0 ] r Vo -t/ol = r Vo — £/ 0 l \Mo t/ 0 ] 

[n 0 VoJ [-^0 M 0 \ ~ [-Afo M 0 \ [iV 0 VoJ 

-[: :] 

We have for some So'. 

G = Go(So) = (Mo + SoU 0 r\No + So Vo) 

= (N 0 + V 0 S 0 )(M 0 + UoSq)- 1 . 



In order to identify So, we will now inject a signal so into the control loop (see 
Figure 3.1), which we assume to be uncorrelated to the input reference signal w\ 
and output disturbance 102 • This yields, according to equation (2.1) 

ro = So«o + M (So) 102 + IV (So) to 1. ( 3 . 2 ) 

Assuming that so is measurable and not correlated to w\ and 102, it follows that we 
can obtain an unbiased estimate for So. Denote this estimate as Sj = N\ — 
M~ l Ni. Our new model for the plant now becomes 

Gi = (Mo + SiUo)~ l (No + Si Vo) 

= (M\ Mo + NiUo)- l (MiN 0 + N1V0) 

= (No Mi + V 0 Ni)(M 0 Mi + UoNi)~ l . 

As in the previous section, it makes sense to update the controller K by design- 
ing a controller Q\ for Si such that the closed loop (Q\, .Sj ) achieves some 
desired objective. In particular, we require that (Qi, So) is also stable. Given 
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Qi = UiV\ 1 = Vj 1 U i and the double Bezout equation 

\m 1 t/ii r y, -(hi = r Vi -£/ii t/ji 

[ni ViJ L-iVi Ml J [-^1 ^1 J l»l y iJ 

-[: :] 

we have that for some Si 

So = St (Si) = (Mi + StUi)- l (Ni + SiVi) 

= (Nt + ViSiXMi + ViS!)- 1 . 

Our new controller becomes 

^i(Gt) = (U 0 + MogiXVo + iVoGir 1 

= (t/ 0 Vi + M 0 Ui)(V 0 Vi + IVot/i)” 1 . 



Also the plant model is now 

G = Gi(Si) = [(iV 0 Mi + V 0 All) + (N 0 Ui + V 0 Vi)5i] 

x [(M 0 Mi + U 0 Ni) + (UoVi + M 0 lh)Si]~ l . 



(3.3) 



(3.4) 



(3.5) 



If the actual closed loop does not respond as hoped for, we can repeat the above 
procedure starting from the system model Gi and controller K \ . 

In order to identify Si, we inject a signal ,v i in the control loop, see Figure 3.2. 
Again, ,vi is generated independently from w\ and W2- We now have 

ri = SJ^i + M(Si)tu 2 + N(Si)w\. 

This allows us to find an estimate for Si. Denote this estimate as S2 = Nj M^ 1 = 
M~ l N 2 etc. 

In this manner we proceed with the identification step of S;, i = 1,2,... fol- 
lowed by the control design step Qi. In the control, we are only concerned with 
the control loop (S;, Qi). The crucial assumption in order to keep nesting the de- 
sign step is that at any one step, (S,_i, Qi) is stable. This amounts to stating that 
we always stabilize the original system. (See Figure 3.3). Clearly, the advantage of 
this nesting approach over the iteration approach in the previous section is that at 
any one step in the design, we have a much easier identification task. What is not 
immediately clear in the present procedure is whether or not we have a potential to 
lose out by a poor design, say at the i th nested loop. Using the iterated scheme, we 
can always recover from a poor /' th iteration design. The following more formal 
derivations serve to clarify the situation, and in particular demonstrate that in us- 
ing nesting, no freedom of design is lost, as long as overall stability is maintained, 
regardless of intermediate identification and/or control mishaps. 
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FIGURE 3.2. Step 2 in nested design 



Equivalent Representation* 

Consider a sequence of models G, e R p for i = 0, ... ,m — 1 . Let AT, e R p 
represent a sequence of associated stabilizing controllers, so that (G;, AT,) are 
stabilizing pairs. Let G, = /V; MJ 1 and AT, = Uj V t ~ 1 , as usual, with all the 
transfer functions /V; , M; , f/ ( - , V,- € for all i = 0 , ... ,m — 1, satisfying the 

double Bezout equation 



~ Mi Ui] 


’ Vi - Ui 1 T Vi 


-Of 


'Mi 


Ui' 


Nj Vi] 


-Nt Mi J ~~ \_-Ni 


Mi 


Ni 


Vt 



-I: :]■ 



For each G e R p there exists a unique S e R p such that 

Gt(S) = (Mi + Gf+iC^G,)” 1 (N t + G i+ i(5)Vi) 
= (Ni + ViG i+l (S )) (Mi + UiGi +1 (S)r l , 



(3.6) 



(3.7) 



for i = 0, . . . , m — 1 with extremes 



G 0 (S) = G, G m (S) = S. 



(3.8) 



Conversely, each S e R p defines via the backward iterations (3.7) a unique G e 

R P . 

It is important to observe that the models G; may be completely arbitrary. Also 
neither S nor G needs to be stable. This is captured in the following two results. 

*This material is more technical than that of Chapters 2 and 3. It is not required for subsequent 
developments. On first reading one need but seek to grasp the key insights without being convinced of 
the detailed formulations. 
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FIGURE 3.3. Step m in nested design 

Lemma 3.1. Suppose we are given a sequence of models Gi £ R p and strictly 
proper stabilizing controllers K, £ R p for Gi, with ( Ni , M, ) and (Uj , V, ) right 
coprime factorizations ofGi and Ki, respectively, i = 0, 1, . . . , m — 1. Then any 
transfer matrix G £ R p can be expressed in the following recursive manner in 
terms of a unique G m (S ) = S e R p : 

G = G 0 (S), Gi(S ) = (Ni + ViG i+ \(S))(Mi + G, G,- , , (.S')) \ (3.9) 

for i = 0. \ ..... m — Moreover, each Gi(S ) belongs to R p . Conversely, any 
given G m = S e R p can recursively yield G m -\ (.S’), ..., Go = G(S) in R p via 
(3.9). 

Proof. See problems. □ 

Notice that for any given plant, there always exists a strictly proper stabilizing 
controller, see Vidyasagar (1985). Hence without loss of generality we can assume 
that JJi is strictly proper. 

For a matrix transfer function G = G(S ) £ R p in the recursive form of (3.9), 
a parameterization of all rational proper controllers for G is summarized by the 
following lemma. 

Lemma 3.2. Given a matrix transfer function G = G(S) £ R p with the recursive 
representation (3.9), assume that Ni, Mi, Uj, Vi e RH 0 0 satisfy the double Be- 
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zout identity of (3.6) with Ni, Ui being strictly proper. Then any rational proper 
controller for G can be recursively parameterized by 

K(Q) = K 0 (Q), KdQ) = (Ui + MiK i+l (Q))(Vi + NiK i+l (Q))-\ 

(3.10) 



for i = 0, 1, . . . , m — 1, in terms of a unique K m = Q e R p . 

The practical importance of these lemmas (Lemma 3.1 and Lemma 3.2) is 
twofold. First, there is no loss of information or control design freedom in the iter- 
ative procedure, since Lemma 3.1 holds for arbitrary G,, A - ,-. This provides strong 
justification for the nested (Q, S) design method. Second, we may expect that 
the iterative design/identification method explained in the previous subsection, 
see also Figure 3.2, leads to stepwise improved models and control performance. 
Each step provides diminishing returns, hence leading to a natural termination of 
the iterative process, which in principle could indeed be continued ad infinitum. 



Stability Results f 

In this subsection, we extend the robust stabilization results for the nested two- 
controller case in Chapter 3 to the nested multicontroller case as depicted in 
Figure 3.3. We present conditions for the multicontroller closed-loop system of 
Figure 3.4 to be stable. Figure 3.4 is a further generalization of Figure 3.3, al- 
lowing us to discuss internal stability more precisely. First, let us consider what 
well-posedness and internal stability means for the multicontroller case. The mul- 
ticontroller scheme of Figure 3.4 is well-posed and internally stable if and only 
if each matrix transfer function from m, to ej exists and belongs to RH 0 0 for 
i,j = l, ... ,2m + 3. We proceed to find the necessary and sufficient conditions 
for this. 

Let us begin with the known case when n — 1 . Consider first the arrangements 
of Figures 3.5 and 3.6. We then have the following mild generalization (to cope 
with the additional external signals) of the stability results of Section 2.2 and 
Section 3.4, see also Francis (1987). 

Lemma 3.3. Given a 2 x 2-block matrix transfer function P of the form (2.2.1), 
assume that P is stabilizable with respect to the controller arrangement of Fig- 
ure 3.5. Then the matrix transfer function from u\, U2, «3 to e\, ei, ej is in RH 0 0 
if and only if K stabilizes /’22- 



Proof. Directly from the definitions of stability. □ 

Theorem 3.4. Consider Figure 3.6 with P being stabilizable with respect to the 
controller arrangement of Figure 3.5. Then the system in Figure 3.6 is stable if 
and only if Q stabilizes S. 
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Proof. Mild generalizations of results in Section 3.4. The key step is to show 
that the subblock of Figure 3.6 with inputs [m u 2 « 3 (e4-M4)]' and outputs 

[ei e 2 e 3 (e 5 -u 5 ) ]' has the form 



Pi(S) = 



PldS) + Pn(.S)M(S)UP 2 i(.S ) P l2 (S)M(S)U P l2 (S)M(S)V 

VM(S)P 2 1 VM(S ) N(S)V 

UM(S)P 2l UM(S ) M(S)V 



P 3 (S)M(S) 
N(S ) 
M(S) 



where 



P(S) = 



Pn(S) 

Pn(S) 



Pn(S) 

F 22 (5)J ’ 



PiiiS) = G(S), 



G(S) = N(S)M(S)~\ N(S) = N + VS, M(S) = M + US. 



□ 

With these results as the background, let us now move to the multicontroller 
case, with first the following lemma, which can be proved by an induction argu- 
ment. 

Lemma 3.5. The stability properties of the control scheme in Figure 3.4 with P 
replaced by P(S) are equivalent to those of the control scheme in Figure 3.7 
for i = \ m. In addition, if P is stabilizable with respect to the controller 
arrangement of Figure 3.5, then Pj,i =2, ... , m are also stabilizable. 

Proof. See problems for an inductive proof. Start with proof results of Theo- 
rem 3.4. □ 

From this lemma, we can immediately see the following result. 

Corollary 3.6. Consider the diagram in Figure 3.4. Then the following relations 
hold: 



02i+2 = Pi,22(S)e2i + 3 + P;, 2 1 



Ml 



+ U2i+2, 



U2i+ 1 



(3.11) 



An important identification implication of the relations (3.11) is that P; j22 (S) = 
Sj(S), i = 0, 1, . . . , m, can be identified successively from measurements. More 
specifically, once the estimate .S',- = NiMf 1 of .§,■ (S) has been obtained through 
identification and a corresponding augmented controller 7, together with Q has 
been constructed, then 5,+i(S), which is a frequency-shaped difference between 

fThis material is more technical than that of Chapters 2 and 3. It is not required for subsequent 
developments. On first reading one need but seek to grasp the key insights without being convinced of 
the detailed formulations. 





FIGURE 3.7. The (m — i + 2)-loop control diagram 

Sj(S ) and Sj, can be further identified on line from measurement. Notice that at 
every stage of the nested design an “open-loop” identification problem has to be 
solved. This is a clear advantage of the nested design over the iterated design of 
the previous section. 

Theorem 3.7. Assume that P is stabilizable with respect to the controller ar- 
rangement of Figure 3.5. Then the multiple control system shown in Figure 3.4 
with P replaced by P{S), see also Figure 3.7, is internally stable if and only if Q 
stabilizes S. 

Proof. By Lemma 3.5, the multiple control diagram is equivalent to the two-loop 
control diagram depicted in Figure 3.7 specialized to the case i = m, P m ^l = 
S m -\ (5) and P m is stabilizable with respect to the controller arrangement of Fig- 
ure 3.5. Thus, applying Theorem 3.4 to this two-loop diagram immediately yields 
Theorem 3.7. □ 

In the special case where S m (S) = 5 = 0, Theorem 3.7 tells us that the nested 
multiple control system of Figure 3.4 is stable provided Q e RH 0 G . This implies 
that the multiple control system is always stable with any given Q e RH 0 0 when- 
ever the mth frequency- shaped plant model error S is sufficiently ‘small’. It is 
also obvious that if Q is both stable and stabilizes S, that is Q strongly stabilizes 
5, then the multiple control system is simultaneously stable for S m (S) = 0 and 
S m (S) = S. 
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Lemma 3.8. Consider the ( m + 1 )-controller strategy of Figure 3.4, see also 
Figure 3. 7 with m = 0, i = l, ... ,2m +3. Assume that Si (0), i = 0, . . . , m — 1, 
are strictly proper. Then the matrix transfer function from ei to e 3 equals K(Q ) 
given as in (3.10) with K m = Q. 

Proof. First, the following relations are immediate from the controller strategy of 
Figure 3.4. 




eim+2 = Qeim+5, i = l ,m. (3.13) 

Let Tj denote the matrix transfer function from e2i+4 to e2i+5- Then it is straight- 
forward to check that (3.12) implies 

e 2 i+3 = Wi + Mi Ti)(Vi + NiTi)-'e 2 i + 2. (3.14) 

Consequently, 

7)_i = (Ui + MiTiXVi + NjTi)~ l . (3.15) 

Noting that (3.13) gives T m = Q, one can conclude that To = Kq with K m = Q. 
This proves the lemma. □ 

Corollary 3.9. Suppose Nj is strictly proper for i =0, \ , ... ,m — 1. Then the 
(m + 1 )-controller scheme of Figure 3.4 is internally stable if and only if the 
matrix transfer function from u\, 112, M3 to e\, 02, e 3 belongs to RH^. 

Proof. We only need to prove sufficiency. Assume that the matrix transfer func- 
tion from m 1 , m 2 , k 3 to e 1 , e2 , e3 belongs to R H 0 Q . Quite evidently, this assumption 
implies the stabilizability of P(S) with respect to the controller arrangement of 
Figure 3.5. From Lemmas 3.8 and 3.3, it follows that K stabilizes G where K is 
defined as in (3.10) with K n = Q. But this is equivalent to Q stabilizing S by 
Lemma 3.2; hence the internal stability of the (m + l)-controller strategy follows 
directly from Theorem 3.7. □ 

Main Points of Section 

This section presents a nested controller structure and a dual nested plant repre- 
sentation. The nested plant representation can be viewed as successive approxima- 
tions to the plant model, and the nested controller as a successive approximation 
to the ‘optimal’ controller for the actual plant. 

The key stability result is a generalization of the results of Chapter 3 for the two 
controller/plant nested representations. Namely the stability of the nested scheme 
depends on the stability of that of the successive designs, based on the successive 
approximations to the plant. 

The nested approach to controller design appears to be a very natural way to 
improve on previous designs. 
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5.4 Notes and References 

The material for this chapter has arisen from the authors earlier works recorded 
in Tay et al. (1989), Yan and Moore (1992), and Yan and Moore (1994). It makes 
connection with the concepts of iterated design as developed in Zang et al. (1991). 
The identification method proposed in Section 5.2 is due to Hansen (1989). It 
is often referred to as identification in dual-Youla parameterization format. The 
method has been extensively used in the context of identification for control (Lee, 
1994; Partanen, 1995; Schrama, 1992b). 

Problems 

1. Verify Theorem 2.2 using the factorizations (2.4.18), (2.4.19) and also for 
(2.4.22), (2.4.23). 

2. Verify the lemmas and theorems of the chapter. 




CHAPTER 6 



Direct Adaptive- Q Control 



6.1 Introduction 

It should by now be evident that the parameterization of all stabilizing controllers 
via a stable matrix transfer function Q is an interesting and powerful control de- 
sign vehicle. In the previous two chapters we indicated how one could optimize 
Q over a number of different control performance objectives, either in a purely 
off-line or iterative identification and control design approach. 

In the off-line method, we start from a given plant model, and our attention is 
focussed on rejecting disturbances or maximizing robustness with respect to un- 
modeled dynamics. In the iterative method, our premise at the outset is that the 
plant model is inadequate and may prevent us from obtaining the desired perfor- 
mance. In this situation, model identification via the dual S parameterization of 
all plants stabilized by a given controller, and control via a ‘plug-in’ controller Q 
is the natural way to proceed. 

In the previous chapter we have considered iterative or nested methods, alter- 
nating identification and control design steps. We demonstrated that, when care 
is exercised, these iterative methods are capable of achieving any desired control 
objective. 

In this chapter, we introduce the first of two adaptive methods. Here, we dis- 
cuss direct adaptive- Q control. In this approach, Q is adjusted without identifi- 
cation of S. This method is suited for the situation where the signal model un- 
certainty is limited. For example, the plant model itself could be adequate but 
the model for the external signals is not, as would be the case when the control 
objective is to track an unknown signal or a signal with uncertain time- varying 
spectral content. Another application of direct adaptive- Q control is for retuning 
an existing controller, either because the original (stabilizing) controller is ineffi- 
cient to obtain good performance or because the design criterion is altered. In this 
chapter we limit ourselves to optimization criteria based on rms signal measures, 
but in principle, the methodology is applicable to any optimization based control 
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design. Whenever control performance is inadequate due to severe model-plant 
mismatch, direct adaptive- Q design is unlikely to be sufficiently powerful, as is 
demonstrated. The analysis of the adaptive - Q method uses averaging ideas. The 
main concepts of averaging as they are required in the analysis are collected in 
Appendix C. 

Unlike traditional adaptive control methods, we start here always from the 
premise that we know how to stabilize the plant, but may not know how to achieve 
the desired control performance. Adaptation is seen here as a mechanism to im- 
prove performance on-line. The analysis we present is aimed at this objective. The 
fact that we use slow adaptation is not a real restriction for our purposes. Indeed, 
adaptive performance enhancement requires adaptation to be slow, at least slow 
as compared to the normal control dynamics of the closed loop. 

The chapter is organized as follows. First we introduce the direct adaptive - Q 
control algorithm. The algorithm is completely developed in state space notation. 
This facilitates the analysis. Indeed, the adaptive- Q control leads to a nonlinear 
and time-varying control system. Despite this observation, and due to the time 
scale separation properties, frequency domain ideas play an important role in un- 
derstanding the behavior of the adaptive system. The direct adaptive- Q method 
is first analyzed under the premise of a perfect plant model. It is then established 
that optimal control is achieved. Next we analyze how the adaptive mechanism 
breaks down under (severe) model-plant mismatch, but with a graceful degrada- 
tion. The generic scheme is discussed. The chapter ends with the discussion of a 
scalar example. 



6.2 Q -Augmented Controller Structure: Ideal Model 
Case 

The plant signal model is represented as: 

Xk + 1 = Axu + Buk + Bw\y, xo, ^ ^ 

yk = Cxk + Duk + W2,k, 

where Xk e "is the state vector, Uk e p is the input, yk e m is the output, w\ t k 
is an input disturbance and W2,k is an output reference signal. The only signals 
available for control purposes are the input uk and the output yk- Neither the input 
disturbance w\y nor the reference signal W2,k are available. The control objective 
is to obtain an internally stable closed-loop system with output regulation, that is 
regulating yk to zero. Otherwise interpreted, we want to have the actual system 
response ( Cxk + Duk ) track the negative of the reference signal u>2,& as closely 
as possible. 

Let us work with a controller structure based on a state estimate feedback ar- 
rangement as depicted in Figure 2.5.4. The controller structure is based on a full 
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state observer. (See Chapter 2 for the development of the state space realization): 
x k +l = Ax k + Bu k - Hr k \ x 0 , 

n = ( ytc ~ Du k ) - Cx k , (2.2) 

Uk = Fxk + Sk- 

Here x+ is the estimate for the state Xk, the estimation residual is rk and Sk is the 
extra input to be generated via the Q filter. The nominal controller (with Q = 0) 
provides stability but may not necessarily attain satisfactory disturbance rejection. 
A nominal controller based solely on the plant model, not taking into account the 
disturbances, could hardly be expected to achieve optimum disturbance rejection. 
The Q filter is introduced to enhance disturbance rejection. In order to obtain an 
implementable adaptively updated Q, we restrict the dynamic complexity of Q as 
follows: 



Zk+ 1 = A q zk + B q r k ; zo, (2 3) 

Sk = ®kZk- 

Here Zk £ Hq is the state of the adaptive Q filter and A q is a stable matrix, 
chosen by the designer. Typically, one would select the eigenvalues of the matrix 
A q to be comparable in magnitude with the eigenvalues of the matrix A + HC, 
which determines the observer dynamics. The output equation is to be adaptively 
adjusted. In this equation, is the adaptation parameter being a pxn, matrix 
of adjustable parameters. 

In order to focus the development, we restrict the desired signal used to formu- 
late the control objectives as ek = [ „ k k ]. Our control objective is to achieve fast 
regulation with disturbance rejection, while containing the control effort. 

As pointed out in (2.5.18), the transfer function 7© from the disturbance signal 
Wk '■= [ Zi’ X ] t0 e k is affine in the transfer function Q. It follows that for &k = © 
(no adaptation), this transfer function is affine in ©. This can also be directly 
observed from the following state space realization for the mapping from Wk to 

e k = [£]: 



Xk+l 




' A + BF 


B&k 


-BF 


B 


0 


Zk + 1 




0 


Aq 


Bq C 


0 


~B q 


Xk+l 


= 


0 


0 


A + HC 


B 


H 


yk 




C + DF 


DQ k 


-DF 


0 


I 


u k 




F 


®k 


-F 


0 


0 



Here Xk = [4 4 ]' is a closed-loop system state variable and the state estima- 

tion error for the plant is Xk = Xk — Xk- It follows from the state space represen- 
tation (2.4) that the closed-loop dynamics are in essence determined by the block 
designed elements A + BF,A q ,A + HC, so that the adaptation parameter Ok 
can not seriously affect the stability of the closed loop. Indeed, BIBO stability of 
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the nominal control design, that is (2.4) with ©£ = 0, informs us that for any 
bounded sequence 0* and any bounded disturbance signal Wk, the state of the 
closed-loop equation (2.4) is bounded. More formally, we can state: 

Lemma 2.1. Consider the system (2.4). Let Wk < W and 1 0* | < 6 for all k. 
Assume that the matrices A + BF, A + HC and A q are stable in that: 



max |eig(A + BF) | < Ai < 1, (2.5) 

max(|eig(A + HC) | , |eig(A ? )|) < A 0 < U < 1. (2.6) 

Then there exist positive constants Co, Ci > 1 such that: 

( tel , fel) < Co^(|xo| , |zol) + C 0 (l - *§)W, (2.7) 

and 

tel < Ci^(|x 0 | + |x 0 | + 9 \z&\) + Ci(l + 6»)(1 - x\)W. (2.8) 



Proof. Follows directly from the variation of constants formula applied to (2.4). 

□ 

Before continuing with the introduction of the adaptive algorithm, it is instructive 
to see how the plug in controller achieves tracking. Let us consider the case in 
which all signals y, u, u>\ and w 2 are scalar valued. From the system equation 
(2.4), it is clear that we obtain a transfer function from uq, W2 to y of the form: 

Li(z) + L 0 (z)L 2 (z) , B l (z) + B @ (z)B 2 (z) 

y = 7) / ( w 2 - (2 - 9) 

Pc(z) Pc(z) 

Here L\, L@, L 2 , Pc, B\, B@, B 2 are polynomials. The polynomial Pc(z) has as 
its roots the eigenvalues of the matrices A + BF, A q and A + HC. The polynomi- 
als L@(z) and B@{z) have coefficients which are linear functions of the © param- 
eters. From (2.9), but also from (2.4), we observe that the poles of the system are 
completely unaffected by the plug in controller 0. However, by appropriate selec- 
tion of 0, we can minimize Hellm^. In particular, if w i , w 2 have a finite spectrum 
containing say n w different frequency lines (counting negative frequency lines!) 
then exact disturbance rejection and tracking can be achieved if © contains at 
least n free parameters which place appropriate zeros in the transfer function. 
Hence we are discussing a notch filter structure, which leads us to an adaptive 
notch filter. 



6.3 Adaptive- Q Algorithm 

In order to highlight the principle underlying Q filter adaptation, we restrict our- 
selves to the simplest of algorithms. Lemma 2. 1 in Section 6.2 greatly simplifies 
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our analysis. Here we are in the pleasant situation where stability is guaranteed a 
priori and so our focus can be on performance issues within an adaptive context, 
rather than on closed-loop stability normally of foremost concern in the analysis 
of adaptive systems. After all, performance enhancement is exactly the main mo- 
tivation for using an adaptive algorithm and the less concern there is on stability 
issues the better. 

Let us adjust ©& as to minimize the criterion: 

1 N 

/(©) = lim — Y^ e ' k Re k \ /? = /?'> 0. (3.1) 

An approximate steepest descent algorithm may be used to update ©£ in a recur- 
sive manner as follows: 



®ij,k+\ — ®ij,k M 



de' k 

d@ij 



Re 1 1 , 

® k 



or in short hand: 



®ij,k+i = ©£ j,k - M Yij,k Re k\ i — 1 • • • P, j = \...n q . (3.2) 

Here ®ij.k is the i jth entry in the matrix ©£, and ytjj is an m + p column vector 
of sensitivity functions, obtained from: 



r tM+i 

Yij.k 



A + BF 


BEij 


C + DF 


D E ij 


F 


Eij . 




r«y ,0 = 0, (3.3) 



where Ejj is a matrix of zero elements except for a unity at the i jth position. No- 
tice that the gradient vector yij,k is indeed independent of (-hj,k, thus confirming 
the earlier observation that the transfer function from Wk to ej is affine in 0. 

In the update algorithm (3.2), the design parameter p. is a small positive con- 
stant which scales the adaptation speed. The equations (3.2) and (3.3) describe the 
“adaptive” mechanism. A priori, (3.2) does not guarantee that ©<■ will be bounded. 
Although, it turns out that in the ideal case, the algorithm does have this property. 
Boundedness may be guaranteed by either projecting ©£ back into a bounded set 
or by introducing some leakage in the update equation (3.2). Projection onto a 
ball (with center 0 and radius 6 denoted as B{ 0, 0)) leads to an algorithm of the 
form: 



®*j,k+\ ~ ®ij,k ~ P'Yij,k Re k , 

®k+l = ®* k +l if ®|+i e B(0, 0), 




otherwise. 



(3.4) 
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Leakage may be implemented as: 

®ij,k + 1 = (1 - M )®ij,k ~ htylj' k Re k , (3.5) 

where A. e (0, 1) is the leakage factor. Leakage contracts & k towards zero, that is, 
it prefers an unmodified controller design. It will be shown that in the presence of 
sufficiently rich signals neither of these modifications is required, at least in the 
ideal case. In general it is good practice to implement either one. 



6.4 Analysis of the Adaptive- Q Algorithm: Ideal Case 

In order to obtain some insight into the behavior of the adaptive algorithm, we 
analyze the closed-loop system described by (2.4), (3.2) and (3.3), not considering 
the projection or the leakage modification. 

Also, in order that the “criterion minimization” task attempted by the adaptive 
algorithm be well posed we assume that the disturbance signal w k is stationary. 
More precisely, we assume that: 

Assumption 4.1. Stationary signals: w k is a bounded signal such that there exist 
constant matrices E w , C w (£), £ = 0,1,... such that for all integers N > 1 

ko+N-\ I 

( w k - E w )\ < 

k=ko 1 

fco+W-i | 

J2 (mwl e -C w (e)y < 

k=k 0 

for some positive constants 2, 3 independent ofko and y e (0, 1). 

From the condition (4. 1 ) it follows that the criterion (3 . 1 ) is well defined for any 
fixed ©. In particular as N goes to infinity the Cesaro mean in (3.1) converges to 
/(©) at the same rate as N~ l+y converges to 0. 

We also assume that the disturbance is sufficiently rich. In particular we assume 
that: 

Assumption 4.2. Exciting signals: w k is such that for any stable, rational, matrix 
transfer function e R the signal wy= w satisfies: 

k 0 +N-l I 

J2 ( u; /l u Vi~ c V ( )'.:S 4 N y , (4.2) 

k=ko 1 

for some 4 > 0, and some symmetric positive definite C Wf , and y e (0, 1). 

Under Assumption 4.2, there exists a unique ©* such that /(©*) < /(©) for 
all ©, indeed /(©) is quadratic in 0 with positive definite Hessian. 



3 N Y , £ = 0, 1, • 
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Remark. Assumption 4.2 not only guarantees the existence of a unique ©* min- 
imizer of /(©) but unfortunately, also excludes the possibility of exact tracking. 
Indeed, Assumption 4.2 implies inter alia that the spectral content of w is not 
finite. 

Under Assumption 4.1 and provided n is sufficiently small we can use averag- 
ing results to analyze the behavior of the adaptive system. For an overview of the 
required results from averaging theory, see Appendix C. The following result is 
established. 

Theorem 4.3. Consider the adaptive system described by (2.4), (3.2) and (3.3). 
Let A + BF, A + HC, A q be stable matrices in that (2.5) and (2.6) are satisfied. 
Let the disturbance signal Wk = [ w' 2 k k ] satisfy Assumptions 4.1 and 4.2, and 
|| uj/t II < W. Then there exists a positive p* such that for all p e (0, p*) the 
adaptive system has the properties: 

1. The system state is bounded; for all initial conditions xq, xq, zo , ©o-' 

|| ©* || < 6 for some 6 > 0, 

(\x k \ , \zk\) < CoAgfljcol , Izol) + C 0 (l - L k 0 )W, 

\x k \ < Ci^(|xol + |%| + Izol 9) + Ci(l + 0)(1 - X\)W, 

for allk = 0,1,... for some constants C\, Co independent of p, W, 9. See 
also (2.7) and (2.8). 

2. Near optimality: 

lim siij)!|%;' — ©*| < Csp^"' 3 '^, (4.3) 

k — >-oo 

where ©* is the unique minimizer of the criterion (3.1). The constant C5 is 
independent of p. 

3. Convergence is exponential: 

§©£ _ ©*j| < c 6 ( 1 - Lp) k |©o - ©*| for allk = 0,1,..., (4.4) 

for some C 6 > 1 and L > 0 independent of p. 

Proof*. Define Xk(@) as the state solution of the system (2.4), denoted A /.(©<.), 
with ®k = ©. In a similar manner define ekf®) as its output. Such correspond 
to the so-called frozen system state and output. These are zero adaptation approx- 
imations for the case when © k ~ 0. Because 0* is slowly time varying and 
because of the stability of the matrices A + BF, A q and A + HC it follows that 
for sufficiently small p, provided |©* | < 9, 

\X k -Xk(@ k )\<C 7 p- some C7 > 0, for all k : |©*| < 9, 

\ek — ek(@k)\ < C 7 p\ some C7 > 0, for all k : | ©* | < 9. 

*This proof may be omitted on first reading. 
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Now, (3.2) may be written as: 

®ij,k + 1 = ®ij,k ~ w'ij l k Re k(®k) + 0{n 2 ), 

which is at least valid on a time interval k e (0, M/fi), for some M > 0. 
The averaged equation becomes: 



®lj,k + 1 — ®ij,k ~ f 1 ~ 



Here we observe that for all finite 0: 



lim 

N^-oo 



1 

N 



ko+N-l 



E 

k=k 0 



Yij,k Re k (. ©) = 



30,7 



This follows from Assumption 4.1. Provided ji is sufficiently small, it follows that 
is bounded and converges to 0*. Indeed, the averaged equation is a steepest 
descent algorithm for the cost function 7(0). In view of Assumption 4.2, 7(0) 
has a unique minimum, which is therefore a stable and attractive equilibrium, see 
Hirsch and Smale (1974). 

The result then follows from the standard averaging theorems presented in Ap- 
pendix C, in particular, Theorem C.4.2. □ 



Remarks. 



1. The same result can be derived under the weaker signal assumption: 

Assumption 4.4. The external signal w is such that there exists a unique 
minimizer 0* for the criterion (3.1). 

Assumption 4.4 allows for situations where exact output tracking can be 
achieved. 

2. Generally, the adaptive algorithm achieves near optimal performance in an 
exponential manner. Of course, the convergence is according to a large time 
constant, as fi is small. 

3. The real advantage of the adaptive algorithm is its ability to track near opti- 
mal performance in the case Wk is not stationary. Indeed, we can infer from 
Theorem 4.3 that provided the signal u>k is well approximated by a station- 
ary signal over a time horizon of the order of l//x, the adaptive algorithm 
will maintain near optimal performance regardless of the time-varying char- 
acteristics of the signal. 

If we consider the adaptive algorithm with leakage, we may establish a result 
which no longer requires sufficiently rich external signals. In this situation, there 
is not necessarily a unique minimizer for the criterion (3.1). The following result 
holds: 
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Theorem 4.5. Consider the adaptive system described by (2.4), (3.3) and (3.5). 
Let A + BF, A + HC and A q be stable matrices satisfying the conditions (2.5) 
and (2.6). Let the external signal be stationary, satisfying Assumption 4.1, and 
|| iVfc || < W. Then there exists a positive p* such that for all p e (0, p*) the 
adaptive system has the properties 

1. The system state is bounded for all possible initial conditions xo, xo, zo, 
®o: 



|| ©* || < 6, forsomeO>Q, 

(1**1 , \z k \) < Co^(|xo| , kol) + C 0 (l - X k )W, 

1**1 < C,a*(|* 0 | + l*o I + kol 0) + C,(I + 0)d - X k )W, 
for all k = 0, 1 , . . . and constants Co, C i > 0 independent of p, W and 6. 

2. Writing dJ/d® = 0 in the form T vec © — E = 0 f ; then there exists a ©*, 
vec 0* = (T + XI)~ l E such that for some C2 independent of p 

limsup I©* — W§i-C2P^ 2 . 



3. Convergence is exponential (for some C3 > 0, L > 0): 

I©* - ©* | < C 3 (l - Lp) k 1 0o - 0*| ; k = 0, 1 



Proof. The proof follows along the same lines as the proof of Theorem 4.3. It suf- 
fices to observe that the averaged version of equation (3.4) governing the update 
for the estimate ®k becomes: 



x / 3/(6) 

\ ^®ij 0 = 0 “” 






The existence of the averages is guaranteed by Assumption 4.1. It follows that 
there exists a unique equilibrium for (4.5) given by ©*. Because F = F 1 ' > 0, and 
T + XI > 0 for all X > 0, then ©* is a locally exponentially stable solution of 
(4.5), that is for sufficiently small p, such that |eig(/ — p(T + a/))| < 1. This 
establishes the result upon invoking Theorem C.4.2. □ 



Remarks. 



1 . The result of Theorem 4.5 establishes the rationale for having the exponen- 
tial forgetting factor (1 — pX) in (3.5) satisfying 0 < 1 — pX < 1. The 
exponential forgetting of past observations should not dominate the adap- 
tive mechanism, otherwise the only benefit which can be derived from the 
adaptation will be due to the most recent measurements. 



dec 0 denotes the column vector obtained by collecting all columns of the matrix 0 from left to 
right and stacking them under one another. 
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2. The minimizers of the criterion (3.1) are of course the solutions of Tvec ©— 
E = 0. In the case of T being only positive semi-definite and not positive 
definite, there is a stable linear subspace of © parameter space, achieving 
optimal performance. In this case the exponential forgetting is essential to 
maintain bounded 0. Without it, the adaptation mechanism will force ©^ 
towards this linear subspace of minimizers, and subsequently ©t- will wan- 
der aimlessly in this subspace. Under such circumstances, there are com- 
binations of w signals that will cause || ©* || to become unbounded. The 
forgetting factor X prohibits this. 

3. The forgetting factor guarantees boundedness, but with the cost of not 
achieving optimality. Indeed, ©* (as established in Theorem 4.5 Item 2) 
solves (T + AJ)ve c© = E, not Tvec© = E. The penalty for 
this is however a small one. If T were invertible, then for sufficiently 
small X, 



vec 0* = T _1 £ - XT~ 2 E + X 2 r~ 3 E + ■■■ 

Hence there is but an order X error between the obtained ©* and the optimal 
one, T-' E. More generally, T and E may be expressed as T = YSi r,T- 
and E = r . a ., w ith r'r* = 1, i = and = 0 

for i / j, i, j = 1 , ,k. The F, may be interpreted as those direc- 
tions in parameter space in which information is obtained from the ex- 
ternal signal w. If k < dim(vec@), that is T is singular, then we have 
that 



A j vec ©* = 0; j = k + l, , dim(vec ©), 

T- vec ©* = — — ; i = 1, . . . , k, 

1 1 + A. 

where the Aj, j — k + 1 , dim (vec 0) complement the F ( , i — 

1 , . . . , k to form an orthonormal basis for the parameter space. Again we 

see that near optimal performance is obtained. 



6.5 ^-augmented Controller Structure: Plant-model 
Mismatch 

In Sections 6.2, 6.3 and 6.4, the ideal situation where the plant is precisely mod- 
eled has been discussed. Normally, we expect the model to be an approximation. 
Let the controller be as defined in (2.2) and (2.3). The nominal controller corre- 
sponds to © = 0. As usual, we assume that the nominal controller stabilizes the 
actual plant. Using the representations developed in Chapters 2 and 3 the plant is 




6.5. Q-augmented Controller Structure: Plant-model Mismatch 167 



here represented by: 



X/c+l 




A 


— HC S 


B 


B 


0 ' 


Vk+1 


- 


—B S F 


A s 


B s 


B s 


0 


yk 




C 


C s 


D 


0 


I _ 



where 



G : 



A B 



C D 



is the realization for the model (see also (2.1)), and 



5 : 



A s B s 
C s 0 



(5.2) 



represents a stable system characterizing the plant model mismatch. The vector 
Vk e ' is the state associated with the unmodeled dynamics. The matrices H 
and F are respectively the gain matrices used to stabilize the observer and the 
nominal system model. The representation (5.1) is derived from the earlier result 
(3.5.1), with the assumption D s — 0 and using a nominal stabilizing controller in 
observer form, that is, with the Z of (3.5.4) having the form 



Z := 




" A + BF 


B 


-H " 


F 


I 


0 


C + DF 


D 


I 



(5.3) 



From the results of Chapter 3, we recall that the controller (2.2) with Sk = 0, that 
is, the nominal controller stabilizes any system of the form (5.1) as long as A s is 
a stable matrix. It is also clear that any system of the form (5.1) with A s stable 
and C s = 0 is stabilized by the controller (2.2) with the stable Q - filter (2.3). More 
importantly it is established in Theorem 3.4.2 that the controlled system described 
by equation (5.1), (2.2) and (2.3) with = © is stable if and only if the matrix 



’ A s B s ©" 

B q C s A q 



is a stable matrix. It has all its eigenvalues in the open unit disk, since this con- 
dition is equivalent to requiring that the pair (Q, S) is stabilizing. It is clear that 
some prior knowledge about the ‘unmodeled dynamics’ S is required in order to 
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be able to guarantee that (Q, S) is a stabilizing pair. (Also recall that this result 
does not require A s to be stable!) 

The closed loop, apart from the adaptation algorithm, may be represented in 
state space format as follows: 



Xk + 1 




" A + BF 


—HC S 


BG k 


-BF 


B 


0 




Xk 


Vkfl 




0 


A s 


B s ® k 


—B S F 


B s 


0 




Vk 


Zk+1 




0 


-B q C s 


A q 


~B q C 


0 


~B q 




Zk 


Xk+l 




0 


0 


0 


A + HC 


B 


H 




Xk 


yk 




C + DF 


Cs 


D@k 


-DF 


0 


I 




W >l,k 


u k 




F 


0 


®k 


F 


0 


0 




_ w 2,k_ 



(5.5) 



The state variable is now x k = {x' k v' k z' k x' k )', where again x k is the state estimation 
error x k = x k - x k - 

Obviously the stability of the above closed loop (5.5) system hinges on the 
stability of the matrix 

T A, fl s 0*l 

l-B q C s A q \ ■ 

Due to the presence of the unmodeled dynamics, it is also obvious that the inter- 
connection between the disturbance signal w k and our performance signal e k = 
[ ] is no longer affine in & k . C s / 0 is the offending matrix. 

In order to present the equivalent of Lemma 2.1 for the system (5.5) we make 
the following assumption concerning the effect of the unmodeled dynamics: 

Assumption 5.1. There exists a positive constant © v such that for all ||©|| < © s 

[ A, Bv©l| 

-B,C S aJ <W0< ‘ <56) 

for some X s > 0. (aq is defined in (2.6)). 

In essence we are requiring that the unmodeled dynamics are well outside the 
bandwidth of the nominal controller, to the extent that for any © modification of 
the controller with “gain” bounded by ® s the dominant dynamics are determined 
by the nominal model and nominal controller system. The bound & s can be con- 
servatively estimated from a small gain argument. In order to obtain a reasonable 
margin for adaptive control, with © iV not too small, it is important that the nominal 
controller has small gain in the frequency band where the unmodeled dynamics 
are significant. 

The small gain argument is obtained as follows. Denote S(z ) = Cs (z I — 
As) -1 Bs and H@(z) = ( zl — A q )~ l B q . Then, the interconnection of (Q, S ) 
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will be stable provided that 

\\®H & (z)S(z)\\ < 1 on |z| = 1, 



or 



II ©II < 



\\H@(z)S(z)\\ 



on |z| = 1. 



(5.7) 



From these inequalities we deduce that in order to have an effective plug in con- 
troller Q, the controller transfer function H®(z ) should be negligible in the fre- 
quency band where significant model-plant mismatch is expected. As explained 
before, due to the frequency weighting, S(z) is only significant outside the pass- 
band of the nominal controller. 

With the above assumption (5.6) we are now in a position to state the equivalent 
of Lemma 2.1 for the system (5.5). 

Lemma 5.2. Consider the system (5.5). Let Assumption 5.1 hold. Let \\wtc\\ < W 
and let condition (2.5) and (2.6) from Lemma 2.1 hold. There exists a A > 0 such 
that for all sequences @k such that || ©*|| < © s and || ©jt+i — ©til < A, the state 
of the system (5.5) is bounded. In particular, there exists positive constants Co, C\ 
such that: 



(\z k \ , Iwjtl , \m < C 0 A.q(|z 0 | , |u 0 | , |* 0 |) + C 0 (l - $)W, 

1**1 < CiA.j(|xol + |xol + |nol + ©i |zol) 

+ Ci(l + ©,)(1 ~x\)W. 

Lemma 5.2 is considerably weaker than Lemma 2.1. Due to the presence of 
the unmodeled dynamics we not only need to restrict the amount of adaptation of 
©s, but also need to restrict the adaptation speed; at least if we want to guarantee 
that adaptation is not going to alter the stability properties of the closed loop. We 
argue that the requirement that adaptation improves the performance of a nom- 
inal control design is essential in practical applications. From this perspective. 
Lemma 5.2 established the minimal (be it conservative) requirements that need to 
be imposed on any adaptation algorithm we wish to implement. It provides yet 
another pointer for why slow adaptation is essential. 



6.6 Adaptive Algorithm 

Let us adjust @4 so as to minimize the criterion (3.1) within the limitation imposed 
by Lemma 5.2. An approximate steepest descent algorithm for updating &k looks 
like: 



®ij,k + 1 = ®ij,k - Wij,k Re k'’ 



i = l...p, 



7 = 1- •- 



(6.1) 
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Here Yij,k is a column vector of approximate sensitivity functions given by equa- 
tion (3.3). When used in conjunction with system (2.4) the Yij,k are asymptoti- 
cally exact estimates for the sensitivity functions, whereas in the presence of the 
unmodeled dynamics Vk, they are only approximate estimates. 

As indicated in Lemma 5.2 the basic algorithm (6.1) needs to be modified us- 
ing projection to guarantee that ||©|| < 0 S and /x needs to be sufficiently small 
so as to guarantee that ||©i+i — ©* || < A. Notice that in essence the adaptive 
algorithm is identical to the algorithm proposed in Section 6.3. 

Before proceeding with the analysis, we provide here the state space realization 
for the exact sensitivity functions with respect to a change in © i; -, denoted gij,k- 
Directly from (5.5) we obtain: 







' A + BF —HC S B@ 


BEij " 








0 Ay B s & 


B s Eij 






Gij,k- i-i 




0 ~B q C s A q 


0 




Gij,k 














gij,k 




C + DF C s D@ 


DE ij 




Zk 






F 0 © 


E U - 





Gy, o = 0. 

Clearly with C s — 0 we recover up to exponentially decaying terms the expression 
for the sensitivity functions as given in equation (3.3). The difference between 
the true nonimplementable sensitivities gijj and the approximate implementable 
sensitivities Yij,k is governed by: 







'A + BF 


— HC S 


B@ 


0 


_ 




0 


A s 


Bs& 


B s Eij 


Gy,* 


= 


0 


~B q C s 


Aq 


0 


8ij,k ~ Yij,k 




C + DF 


C s 


D@ 


0 






F 


0 


© 


0 



Gy, o = 0. (6.3) 

From the above expression it follows that the implemented sensitivity function 
Yij,k is a good approximation for the exact sensitivity function gijk under the 
now familiar conditions: unmodeled dynamics are effective only outside the pass- 
band of the nominal controller, and the disturbances Wk to be rejected are limited 
in frequency content to the passband of the nominal controller. Notice also that 
the linear gain from zk to gi^k — Yij,k may be over-estimated by a constant propor- 
tional to C v | , which again confirms that for C s = 0 the approximate sensitivity 
function is asymptotically exact. 
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6.7 Analysis of the Adaptive- Q Algorithm: 
Unmodeled Dynamics Situation 

In this section, in studying the unmodeled dynamics situation, we proceed along 
the same lines used for the analysis of the adaptive algorithm under ideal model- 
ing, see Section 6.4. In particular we assume that the external signals w k are both 
stationary and sufficiently rich. 

Unfortunately, due to the presence of the unmodeled dynamics we can no 
longer conclude that /(©) is quadratic in 0. Indeed, we have for © constant, 
using Parseval’s Theorem: 



1=1 

l r 2n . fl ■(, -o 

= — / W*(e ,e )T*(e ld )RT & (e ie )W(e ,e )dO. 
27r Jo 



Here the transfer function from the disturbance Wk = [ "^1] t0 e k — [«!]. de- 
noted 7©, has a state space realization as indicated in (5.5) and W(z.) is the z- 
transform corresponding to the external signal w. Obviously 7© is not affine in 
©, hence /(©) is not quadratic in ©. 

Under these circumstances, we can establish the following result: 



Theorem 7.1. Consider the adaptive system described by (3.1), (3.2) and (5.5). 
Let 9 < ©5. Assume that the conditions (2.5), (2.6) and (5.6) are met. Let the 
external signal w satisfy the Assumptions 4.1 and 4.2. Then there exists a p.* > 
0 such that for all p. e (0, pf) and ||©oll < 9 and all xq, xq, zo and vq . the 
system state is bounded in that || ©* II < ©s and Lemma 5.2 holds. Consider the 
difference equation 



® a ij,k + 1 — ®ij,k ~ M - 



with e = || Cs || and 



bij ( ©) = ^lim^ ^ ~ Yij,k{®))'Re k m, 



(7.3) 



where gij,k(@) — Yij.ki®) is described in (6.3) and 



«*(©) i 



M®) 

_u k (@)_ 



(7.4) 



follows from (5.5) with = ©• Provided (7.2) has locally stable equilibria 
©* € B(Q, ©s), then @ k converges for almost all initial conditions || ©oil <9 to 
a p,^-®/ 2 neighborhood of such an equilibrium. 
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Proof. Follows along the lines of Theorem 4.3. □ 

Equation (7.2) is crucial in understanding the limiting behavior of ©£. If no lo- 
cally stable equilibria exists inside the ball 5(0, &s), a variety of dynamical be- 
haviors may result, all characterized by bad performance. The offending term in 
(7.2) is the bias term (?, ; (©). Provided ebij (©) is small in some sense, good per- 
formance will be achieved asymptotically. 

As explained, the bias bij (©) will be minimal when the disturbance signal has 
little energy in the frequency band of the unmodeled dynamics. One conclusion 
is that the adaptive algorithm provides good performance enhancement under the 
reasonable condition that the model is a good approximation in the frequency 
band of the disturbances. 

Notice that these highly desirable properties can be directly attributed to the 
exploitation of the Q -parameterization of the stabilizing controllers. Standard im- 
plementations of direct adaptive control algorithms are not necessarily robust with 
respect to unmodeled dynamics! (See for example Rohrs, Valavani, Athans and 
Stein (1985) and Anderson et al. (1986).) 

The above result indicates that the performance of the adaptive algorithm may 
be considerably weaker than the performance obtained under ideal modeling, 
embodied in Theorem 4.3. Alternatively, Theorem 7.1 explores the robustness 
margins of the basic adaptive algorithm. Indeed, in the presence of model mis- 
match, the algorithm fails gracefully. Small model-plant mismatch (small e) im- 
plies small deviations from optimality. Notice that the result in Theorem 7.1 re- 
covers Theorem 4.3, by setting e = 0! However, for significant model-plant mis- 
match we may expect significantly different behavior. The departure from the de- 
sired optimal performance is governed by the bij (©) term in (7.3). It is instructive 
to consider how this bias term comes about. A frequency domain interpretation is 
most appropriate: 

M«) = lF*(e^)r* s _ K) ..( e ie )r* z (e ie )5r w ( e i0 ) W{e w )dO, 

where T^g-y)^ has a state space realization as given in (6.3) and T wz , T we have 
state space realizations given in (5.5), and are respectively the transfer functions 
from r to (g — y)ij, w to z and w to e. 

In Wang (1991) a more complete analysis of how the adaptive algorithm fails 
under model plant mismatch conditions is presented. It is clear however that under 
severe model mismatch the direct adaptive Q mechanism is bound to fail. Under 
such conditions reidentification of a model has to be incorporated in the adaptive 
algorithm. This is the subject of the next chapter. 

Example. Having presented the general theory for direct adaptive- Q control, we 
develop now a simple special case were the external signal, w\ = 0, wzk = 
cos o)\k and the adaptive- Q filter contains a single free parameter. Whereas the 
development of the general theory by necessity proceeded in the state space do- 
main, due to the slow adaptation the main insight could be obtained via transfer 
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FIGURE 7.1. Example 

functions and frequency domain calculations. For the example we proceed by 
working with transfer functions. 

Consider the control loop as in Figure 7.1. All signals u,y,r and .v are scalar 
valued. Let the control performance variable be simply e = y. We are thus inter- 
ested in minimizing the rms value of y\ 



J(6)= lim -^yf(0). 



The plant is given by 



Njz) + S(z)V(z 
M(z) + S(z)U(z )' 



The controller is given by 



U(z ) + 6H(z)M(z) 
V{z) + eH(z)Niz ) ' 



with Miz)Viz) - N(z)U(z) = 1; N, S, V, M,U,H e RH a e . 
For constant 0 we have 



y k (0) = - 



u k iO) = - 



jMjz) + S(z)U(z))(V(z) + 0H(z)N(z)) 
1 - S{z)6H{z) 

Mjz) + SjzWizmUiz) + OHjzWiz)) 
1 - S(z)0H(z) 



cos wik, 
coscoik, 



Yk(0) = H{z)N{z){Miz)ym - Niz)u k iG)). 



(7.5) 
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The update algorithm for 0 (with exponential forgetting X e (0, 1)) is thus given 
by (compare with (3.5)) 

0k+ 1 = (1 - M Wk ~ HYkyk- (7.6) 

Following the result of Theorem 7.1 the asymptotic behavior of (7.6) is governed 
by the equation 

Ci = (! - M)C - WKC)- (7.7) 

where g{0) is given by 

1 N 

g {0)= lim TnWykiO). 

N^-oo N “ 



When S(z) = 0, that is the ideal model case, g(0) can be evaluated using (7.5) as 

gi(0) = (ReV(e~ j(01 )H(e j(0l )N(e jmi ) +o\H(e jm )N(e im )\ 2 y 

(7.8) 



where the index i reflects the situation that S(z) = 0. 

In general we obtain the rather more messy expression: 



= | M(eM) + S(eM )t /( e M) | 

| 1 -S(ej“>')H(e j,o ')0 | 

• (Re jff(e-- / '® 1 )^V(e“ M )y(e- 7 '® 1 )) + (9 



(7.9) 

)N(e >')| 2 ^ . 



Let us discuss the various scenarios described in, respectively, Theorems 4.3, 4.5 
and 7.1 using the above expressions (7.8) or (7.9). 



1. Ideal case: X = 0 (Theorem 4.3), expression (7.8). 

There is a unique, locally stable equilibrium; g (0 * ) = 0, or 

0 „ « _ R e{H(e-M)N(e-M)V(eM)} (? 1Q) 

|ff( e -M)At( e -M)| 2 

This equilibrium achieves the best possible control. The adaptation approx- 
imates this performance. Indeed, it is easily verified that 



J(0*)<J(O) for all 0. 

In this case the performance criterion ||y || 2 = J{0) is given by 

II y II ms = |M(e>‘)| 2 | V(e>') + 0H e (e^')N(e^)\ 2 . 

In general, we can not expect to zero the output, unless the actual signal 
happened to be a constant wi = 0, in which case we indeed obtain 0* = 
-V(1)/(H(1)N(1)) and J{0*) = 0. 
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2. Ideal case: k ^ 0 (Theorem 4.5). 

Again there is a unique, locally stable equilibrium, now given by 

7 + |M(e>i)| 2 |ff 0 (eM)//(eM)| 2 

In this case, the performance of 9* is no longer the best possible, but clearly 
for small 7, we have that |d* — 9*\ = 0(k), see (7.10) and (7.11). 

3. Plant-model mismatch: 7 = 0 (Theorem 7.1). 

Remarkably, there is again a locally stable equilibrium 9*, but also there 
may exist an equilibrium at oo as g(9) — >• 0 for Q —*■ ±oo. Clearly in the 
presence of unmodeled dynamics, the adaptive algorithm loses the property 
of global stability. 

Despite the fact that 9* is always a locally stable equilibrium of (2.2), it may 
not be an attractive solution for the adaptive system. Indeed Theorem 7.1 
requires that the closed-loop system (S(z), Hf,(z,)9) be stable, a A property 
that may be violated for 9 = 9* if S(z ) is not small. Theorem 7.1 requires 
at least that \S(e^ COl )H (e-’ < ° 1 )9*\ < 1. If this is not the case, the adaptive 
system will undergo a phenomenon known as bursting. See Anderson et al. 
(1986) or Mareels and Polderman (1996) for a more in-depth discussion of 
this phenomenon. Let it suffice here to state that whenever 9 *, the equilib- 
rium of (2.2), is such that (S(z), H{z)9*) is unstable, the adaptive system 
performance will be undesirable. 

The performance of 9* is also not optimal with respect to our control crite- 
rion. When S(z ) 0, the criterion becomes: 

2 \V(ej (01 ) + 9H(ej (0l )N(ej (0l )\ 2 \ + S(e> 1 )t/(e> 1 ) | 2 

I - )H{ej"')0 2 

It can be verified that the performance at the equilibrium 9* will be better 
than the performance of the initial controller 9 = 0 if and only if 

Re { N(e~ j<ai )V(e jwi )H(e jmi )j Re { S(e Ja,1 )H(e jt0 ')} 

> -- ^Re{A(e _M )V , (e M )77(e M )}) 2 

W 1 | S(e M ) | 2 \ 

' \|V(eM)| + |iV( e M)| ) ■ 

This condition is always satisfied for sufficiently small | S (e- 7 " 1 ) | . The above 
expression for this example is the precise interpretation of our earlier ob- 
servation that the direct adaptive Q filter can achieve good performance 
provided S , the plant-model mismatch, is small in the passband of the con- 
troller and provided the external signals are inside the passband of the nom- 
inal control loop. 
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4. Plant-model mismatch: A ^ 0. 

Due to the presence of A as well as model-plant mismatch, we now end up 
with the possibility of either 3 or 1 equilibria. Indeed the equilibria are the 
solutions of X0 + g{9) = 0 which leads to a third order polynomial in 9. 
For small values of A. >0, there is a locally stable equilibrium 0* close 
to 9*. Global stability is of course lost, and the same stability difficulties 
encountered in the previous subsection persist here. 



6.8 Notes and References 

Direct adaptive- (2 control has been studied in some detail in Wang (1991), build- 
ing on the earlier work of Tay and Moore (1991) and Tay and Moore (1990). The 
paper Wang and Mareels (1991) contains the basic ideas on which the present 
theory is built. Most of the material in this chapter has not been published before. 

For a more complete presentation of averaging techniques in continuous-time 
setting, we refer the reader to Sanders and Verhulst (1985). Averaging ideas have 
been used extremely successfully in the analysis of adaptive systems. We refer 
the reader to Anderson et al. (1986), Mareels and Polderman (1996) and Solo and 
Kong (1995) for more in depth treatises. Much of the presentation here could have 
been presented using a stochastic framework. The book of Benveniste, Metivier 
and Priouret (1991) is a good starting point. The treatment of the actual dynam- 
ical behavior of adaptive systems has, by necessity, been superficial. One should 
not lose sight of the fact that in general, an adaptive scheme leads to a nonlin- 
ear and time-varying control loop, the behavior of which is all but simple. We 
have identified conditions under which the adaptive system response can be un- 
derstood from the point of view of frequency domain ideas using the premise that 
the adaptation proceeds slowly compared to the actual dynamics of the controlled 
loop. Whenever this condition is violated, adaptive system dynamics become dif- 
ficult to comprehend. The interested reader is referred to Mareels and Polderman 
(1996, Chapter 9). 



Problems 

Problems 1, 2 and 3 are intended to give the reader some insight into averaging. 

1. Consider 

x k+ i = (1 - ap,)x k + /x(l + cosk). (8.1) 

Here xk a, pt are scalar variables, /x > 0 and small, a e . Also, consider 
the ‘averaged’ difference equation 



xi+i = (1 - ap)xl + /x. 



( 8 . 2 ) 
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(a) Compare the solution of (8.1) and (8.2), both initialized with the same 
initial condition xq. Show in particular that x* — x" < C/j., k = 

0, 1 L/ji for any L > 0 and some constant C independent of fi, 

but dependent on L. 

(b) If a < 0, show that the error \xk — x^ < Cji for all k. 

In a sense, averaging is a technique that formalizes the ‘low pass’ charac- 
teristics of a difference equation of the form x^ + \ = x/. + tifk(xk). 

2. Show that any signal of the form E^j cos coik has a uniform zero average. 

3. Show that any signal u>k that converges to zero has zero mean. (Averaging 
eliminates transients!) 

The following problem illustrates the theory of direct adaptive- Q design 
and allows the reader to venture beyond the theory through simulation. 

4. Refer to Figure 7.1. Let 



G(z) = 



K{z) = 



2-V2 z+l 
2 z 2 - V2z + 1 
_J__ , 1+V2 
~^/2 Z+ ^W2 2 






Let the signal to be tracked be cos((7r/4)k), cos 1.75k or cos((jr/4)k) + 
cos 1.75k. Explain for each of these reference signals the performance that 
the adaptive algorithm achieves; use a single scalar adaptive 0. Let /i — 
0.01, Hq(z) = z~ x . (One could also utihze Hf t (z) = 1 in this case). Con- 
sider the following questions: 



(a) Show that the original controller achieves dead beat response. 

(b) Show that the original controller has perfect plant output tracking for 
any signal of the form A cos((7r/4)k + cp). 

(c) Show that the adaptive- Q filter will not deteriorate the performance 
of the loop (if /z is small). 

(d) Despite the fact that a single 0 can not achieve sinusoidal tracking, 
show that significant improvement is obtained for either u>k = 
cos 1.75k or Wk = cos((?r/4)k) + cos 1.75k. 

(e) Using averaging ideas, find the optimal performance that can be 
achieved for Wk = cos wk. (That is, plot J (9*) as a function of co e 
(0, 7t)). What do you learn from this plot? 

(f) Using the adaptive Q(z) = 0\ + 02 z -1 , show that exact tracking can 
be achieved for Wk = cos((7r/4)k) + cos wk, any co. 
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(g) Introduce S(z ) = rz _1 . show that this amounts to a significant per- 
turbation in the low frequency response; that is, this is a significant 
perturbation, and in particular, the plant’s resonance disappears. 

(h) Continuing with the scenario ending Problem 4g, introduce forgetting 
and use a scalar adaptive Q(z) = 0. Find the equilibria as a function 
of X, and r for Wk = cos 1.75 k. 

(i) When does the closed-loop adaptive system lose stability? This last 
part should not be attempted unless you have a few spare hours. A 
complete analysis comprising (a complete range of) X, r has never 
been completed, let alone an analysis considering ranges of the pa- 
rameters ii,X, x and to. 




CHAPTER 7 



Indirect (Q, S) Adaptive 
Control 

7.1 Introduction 

Direct adaptive- Q design is applicable in those situations that a reasonably good 
model for the plant dynamics is available. It is geared towards tuning of con- 
trollers, and in particular for tuning controllers for new design criteria without 
sacrificing stability, and also towards disturbance rejection. In some situations, 
our limited knowledge of the plant dynamics may be the greatest obstacle on the 
route to better control performance. In such situations, the iterative and nested 
(Q, S) designs are needed. 

In this chapter, we will present an adaptive version of the nested (Q, S) method- 
ology in that identification and control are both adjusted on-line as new data be- 
comes available. Adaptive methods where the controller design is based on an 
on-line identified model are often referred to as indirect adaptive control meth- 
ods. Hence the title, indirect adaptive (Q, S) design. 

In the present case, where the lack of a sufficiently accurate model of the plant 
is the major issue in obtaining satisfactory control performance, we need to gather 
information about the plant. In the spirit of the book, this is achieved by identifica- 
tion of an S model. As before, we assume that a stabilizing controller is available, 
and hence we can parameterize a class of systems which contains the plant. In or- 
der to identify the S parameter and thus the actual plant on-line, in a closed-loop 
control context, we inject an external signal into the closed loop at the output of 
the plug-in controller Q. This probing signal is required to gain information about 
S and as a consequence, will necessarily frustrate to some extent any control ac- 
tion for as long as it is present. This is the penalty we must be prepared to pay for 
our lack of knowledge of S and our desire to obtain better control performance. 
As pointed out before in Chapter 5, identification of S in closed-loop operation 
is nontrivial. The methods proposed in Chapter 5 to circumvent the problem in 
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iterated- Q design are not particularly well suited for an adaptive- Q approach. In- 
deed, these methods invariably rely on the time invariance of Q, a property which 
is lost by the very nature of an adaptive design. One solution could be to use two 
time scales in the adaptive algorithm; one, the fast learning time scale, for the 
adaptation of a model for S and a second much slower time scale for the adap- 
tation of Q. This way, we would recover more or less the same behavior as in 
the nested design. If desired, one could alternate time scales between S identi- 
fication and Q design at preset intervals to recover a full adaptive version of a 
multiple nested (Q. S) design method. The idea has been worked out in detail, 
and analyzed using averaging techniques in the PhD thesis of Wang (1991). 

Here we explore a different method for the case where plant-model mismatch 
S is significant only in the frequency range above the passband of the nominal 
control loop. Probing signals are thus best utilized if their frequency content is 
located around and past the cut off frequency of the closed loop. The probing 
signals will only marginally affect the control performance. In order for Q to be 
effective, it has to shape the response in the same frequency range where S is 
important. The presence of Q frustrates the identification of S in so far as the 
probing signals affect Q. The idea is to augment Q with a filter at its input that 
filters out the probing signals. Of course, this limits our control ability, but ensures 
good identification, and thereby control response improvement. These ideas are 
worked out in this chapter, making use of ^-optimization design criteria, as these 
fit most naturally with the averaging techniques for the system analysis. 

The chapter is organized as follows. First we discuss the basic framework. As 
in the previous chapter to facilitate our analysis techniques, the development pro- 
ceeds in the state space framework. Next we discuss the adaptive mechanisms 
and present some results exploring the time scale separation between the adaptive 
mechanism and the control dynamics. The chapter is concluded with a discussion 
of an example. 



7.2 System Description and Control Problem 
Formulation 

Our starting point is a stable plant-controller configuration which is characterized 
by unacceptable performance due to an inaccurate nominal plant model for the 
controller design. Again G represents the nominal plant, K the nominal controller, 
G the actual plant, S embodies the plant-nominal plant mismatch and Q is the 
plug-in controller to be designed. 

Since the nominal controller K stabilizes G, the mismatch system S is stable. 
For estimation purposes, we introduce the following parameterized class of stable 
rational transfer functions: 

N 



J2ZiBi(z)=3B(z). 



( 2 . 1 ) 
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The Bj (z) form a collection of basis functions, being stable rational transfer func- 
tions. The precise choice of (z) should reflect any prior knowledge of the plant 
uncertainty S(z ). The matrices 3,-, collected into the coefficient matrix 3, have to 
be estimated from on-line data. 

We refer to the situation where S(z ) is indeed of the form (2.1) as the ideal 
case. In this situation there exists a unique coefficient matrix 3* such that S(z ) = 
3 *B(z). When no such representation exists we speak of the nonideal case. For 
future reference, let B{z) possess a state space realization: 



B(z) : 




( 2 . 2 ) 



with A s a stable matrix. As indicated before, the dominant eigenvalues of A s must 
be compatible with the closed-loop bandwidth of the system (G, K). 

In the ideal case we have for S (z) 



S(z): 



A s B s 
3 *C S 0 



In the nonideal case we represent S (z) as 







0 


Ba " 


S : 


0 




B s 




L Ca S*C s 


0 



(2.3) 



(2.4) 



Here A a is a stable matrix, and ||Ca|| can be considered as a measure of nonide- 
alness, being zero in the ideal case. 



Remark. A linear parameterization for S (z) such as 3 *B(z) may not be the most 
economical way to parameterize the unmodeled dynamics. A rational fraction 
parameterization may require quite fewer parameters, but has the difficulty that 
somehow the parameters must be restricted to represent a stable S(z). This is 
nontrivial, as the set of stable rational transfer functions does not have a simple 
representation in parameter space. The present approach avoids this complication 
and gives transparency to the associated performance analysis. 



Let the nominal plant G be represented 



x k+ i = Ax k + Bu k + Bwiy, x 0 , 
yk = Cxk + Duk + W2,k, 

where w\^ is an input disturbance, W 2 ,k is an output disturbance. Also, x k e ", 
Uk e p , yk e m . The nominal controller K is represented as: 

h+\ = Ax k + Bu k - Hr k \ x 0 , 

n = yk- ( Cx k + Du k ), 

u k = Fxk + s k , 
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where r k is the observer innovation and su is the auxiliary control signal. 
The plug-in controller Q takes the special form: 

Zf,k + 1 = A fZf,k + B f r k\ Zf, 0, 

r f,k = C f Zf,k , 

Zk + 1 = A q Zk + A-kZk + BqffX, ZO, 

Sk = ®kZk + dk- 



The signal dk is an external signal added to aid identification of E. The parameter 
matrices A k and @k are to be updated adaptively on the basis of E*, the present 
estimate of E*. As earlier, the remaining Q parameters are chosen a priori. The 
matrices Af,A q are stable. The system ( Af,Bf,Cf ) is designed to have the 
specific property that the steady state response of the filter subject to the input dk 
as input is zero. Thus for the system 



z/,k + 1 = AfZf.k + Bfdk, 
df,k = CfZf.k, 



]imk^.oodf,k = 0. As far as the probing signal goes, the controlled system looks 
like an open-loop system. 

Finally, the actual plant G, can now be represented as: 



Xk + 1 




A 


-hc a 


-Ha*C s 


B 


B 


0 " 


fA.fc+1 




-BF 


Aa 


0 


Ba 


Ba 


0 


l^+l 




—B S F 


0 


A, 


B s 


B s 


0 


yk 




C 


C A 


s *c s 


D 


0 


/ 



The complete closed-loop equations are: 



Xk+l ■ 




' A + BF -HC A 


-HE*C S 0 


BS k 


BF 


B 


0 B 




v A,k+ 1 




0 A a 


0 0 


B A ®k 


b a f 


B 


0 B 




v k+ i 




0 0 


A s 0 


B s @k 


B S F 


b a 


0 B A 




Zf,h+ 1 




0 -B f C A 


—B f E*C s A f 


0 


B f C 


o 


-B f 0 




Zk+1 




0 0 


0 B q C f 




0 


0 


0 0 




4+i 




0 0 


0 0 


o 


A + HC 


B 


HO 




yt 




C + DF C A 


S *C S 0 


D@ k 


DF 


0 


I D 




“* - 




F 0 


0 0 


@ k 


F0 


0 


/ 





In order to be able to develop a meaningful adaptively controlled closed loop 
we impose the following assumption. 
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Assumption 2.1. The plug-in controller structure is such that for almost any S 
the eigenvalues of the matrix 



-B f uC s A f 

0 BqCf 



B s © 
0 



can be assigned arbitrarily by the appropriate selection of the matrices © and 
A. (Arbitrary eigenvalue assignment under the restriction that for any complex 
eigenvalue its complex conjugate is also to be assigned.) 

Remark. Assumption 2.1 implies generically that we have enough freedom in 
the controller structure for pole assignment. That is the best we can hope for. 

The system matrix (2.8) corresponds to the interconnection of ( Q , S ), assuming 
that S belongs to the model class defined by (2.1). 

A sufficient condition for arbitrary pole assignment of A c is that the matrix pair 



r a, on m 

L ~B f ZC s A f J |_0 J 



be controllable, the matrix pair 



As ° 1 , |° B q Cf\ 
1-BfZC, A f \ V q f \ 



be observable and that the dimension of A q be at least as large as dim A s +dim A /. 
In this case, we could use the following special choices: 



'I 



0 Ai 2 BqCf 

—BfaC s AnBqCf 

Hence A 12, A22 are chosen such as to arbitrarily assign the eigenvalues of A q + A, 
and 0 chosen to place the eigenvalues of 



f A * °1+Kb 

-B f EC s A f \ |_0 



Clearly the above observability and controllability conditions are generically sat- 
isfied. 

To complete this section, we now formulate the particular indirect adaptive 
control problems we discuss. 
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Problems 

1. Adaptive pole assignment. 

Consider the system (2.7). Let C a = 0. Assume that all matrices are known 
except for 3*. The matrices A + BF, Aa, A s , Af, A + HC and A q are 
stable. Assume that for 3 = 3* the eigenvalues of A c in (2.8) can be 
arbitrarily assigned by appropriate selection of 0 and A. Assume that the 
external signals w i tk , W2,k and d k are stationary and mutually uncorrelated. 
The indirect adaptive control objective is to design a controller using the 
information available in closed loop, being the signals y*, u k , Zk, r k> s k and 
dk such that asymptotically the eigenvalues of A c (3*, 0, A) are placed at 
preassigned stable locations. 

2. Adaptive LQ control. 

Consider the system (2.7). Let Ca = 0, being the ideal S model case. 
Assume that all system matrices are known except for 3* and that the 
matrices A + BF, A + HC, Aa, A s , Af, and A q are stable. Assume 
that for the plant G, with 3 = 3*, the eigenvalues of (2.8) can be 
arbitrarily assigned by appropriate selection of 0 and A. Assume that 
the external signals W2,k and dk are stationary and mutually un- 
correlated. Design an adaptive controller such that the performance in- 
dex 



7(0, A) = lim - (r' k r k + s' k s k ) 



is minimized. 

Remark. 

Both problems have received a lot of attention in the adaptive control literature. 
The important distinction with the classical literature on these topics is the starting 
assumption of an initial stabilizing controller. This brings a lot of structure to the 
problem. This structure has been exploited to the maximal extent possible in our 
system description (2.7). 

In setting up the adaptive control problem the designer has much freedom to ex- 
ploit prior knowledge about the system dynamics. The most important instances 
of this are the appropriate selection of B(z), the spectrum of the probing signal d k 
together with the associated filter ( Af,Bf,Cf ). 



Main Points of Section 

Starting from the assumption that a stabilizing controller is available for the 
plant to be controlled, we provided a complete description of the closed-loop 
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system, suitable for adaptation (2.7). The mismatch S between the particu- 
lar nominal plant G and the actual plant G has been represented via a pa- 
rameterized class of stable transfer functions with parameter E. The corre- 
sponding parameterized class of plug-in controllers Q, with parameters (©, A), 
has special structure to ensure a stable (Q, S) and to filter out the prob- 
ing signal d used for identification of S, or E. The control objective is to 
either obtain pole placement for the loop ( Q,S ) or achieve an LQ control 
goal. 

In the next few sections we introduce the adaptive algorithms, which will be 
based on the identification of E*, and then continue with their analysis. 



7.3 Adaptive Algorithms 

As indicated, we proceed with the introduction of an indirect adaptive control 
algorithm. First an estimate for E* is constructed, denoted E*. This estimate is 
then used in a typical certainty equivalence manner to determine the control pa- 
rameter ©yt and Afc. Given an estimate E*, there exists a mapping (E© that 
determines ©&, A/.. The mapping reflects the particular control objective of 
interest. The problem is that is not continuous on all of the parameter space. 
In the event that the estimate E* leads to a system description for which either 
the pair (2.9) fails to be controllable, or the pair (2.10) fails to be observable, 

may not be well defined. This is a manifestation of the so-called pole/zero 
cancellation problem in adaptive control. Either event is rather unlikely, but not 
excluded by the identification algorithm we are about to propose. Here, as is tra- 
ditional in adaptive control, we simply make the assumption that the above event 
will not occur along the sequence of estimates E* that the adaptive algorithm 
produces. 

Assumption 3.1. Along the sequence of estimates E*, k = 0, 1, . . ., the matrix 
pair (2.9) is controllable and the matrix pair (2.10) is observable, in that the 
smallest singular value of the controllability matrix, respectively observability 
matrix, is larger than some constant a > 0. 

We propose the following adaptive control algorithms: 

1. Filtered Excitation Algorithm. 



fo+i = AsVk + B s @kZk + B s dk\ vo, 

z/,k + i = A fz/,k - BftLkCsVk ; zf, o, 

gk + 1 = A Sgk + B s dk\ go = 0 , 

Yij,k + 1 = A fYij,k - BfEijCggk; H;',0 = 

&ij,k- 1| = &ij,k ~ b-(Zf,k ~ Z f,k)' Yij ,k\ So, 

(©*,A©= (E *). 



(3.1) 
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2. Classic Adaptation Algorithm. 



1 

Zf,k + 1 
Yij,k + 1 



A s v k + B s ® k zk + B s d k \ v 0 , 

A fif,k ~ Bf^kCsVk; Zf, o, 

A fVij,k ~ B f EijC s v k ; yij , 0 = 0, 

B . . k _ ---- ^ - . 

1 + y (z/,* - Z f,k)'(z f,k - Zf,k ) + y Yij,kYij,k ’ 



(0*,A*) = (Si). 



(3.2) 



Remarks. 



1. In the filtered excitation algorithm, the presence of the probing signal dk 
is essential. If dk = 0 then there is simply no adaptation. This provides 
an interesting mechanism to switch on or off the adaptation. By simply 
removing the probing signal the controller becomes frozen. 

2. In the filtered excitation algorithm the gradient vector y ( - is only affected 
by the probing signal itself. The intention is that the other signals will be 
orthogonal to it, hence on average not affect the identification at all. Due to 
the particular set up involving a probing signal with an associated filter in 
the Q loop, the identification process is on average unbiased and similar to 
an open-loop identification process. This will be demonstrated. 

3. It is not a good idea to start with 3 q = 0. In this case Assumption 3.1 is 
automatically violated. An alternative is to have (©£, A k) = 0 until such 
time that S k satisfies Assumption 3.1. This is a good method to start the 
adaptive algorithm since for (©/., A© = 0 the closed loop is stable by 
construction. 



4. The update algorithm for 3^ is a typical least squares algorithm. The step 
size /i > 0. In the literature one often finds a normalized update algorithm: 



&ij,k+ 1 = &ij,k ~ I* 



(z'f,k ~ z f.k) Yij.k 
1 + Yij,kYij,k 



In the filtered excitation algorithm this normalization is superfluous because 
Yij k is bounded by construction, regardless of the state of the system. In- 
deed the normalization is only necessary when it is not clear whether the 
gradient Yij,k is bounded or not, which is the case when is determined 
by 



Yij,k + 1 = A fYij,k ~ BfEjjC s Vk. 



(3.3) 
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This is the classical update algorithm. It has the advantage of not requiring 
a probing signal. In the ideal case one can even demonstrate that it will 
suffice to solve a weak version of Problem 2. Nevertheless, (3.3) introduces 
some nontrivial nonlinearities in the identification and control loop, which 
lead to biased estimates for E*. The algorithm in (3.1) circumvents this 
problem altogether. The classical algorithm allows for 0 < p, < 1. Here we 
restrict ourselves, as before, to small values of //, that is slow adaptation. 

5. A poor selection of Eo, with associated (©o, Ao) = (So) may well lead 
to an initially destabilizing control loop. Our analysis will not deal with 
this situation explicitly, although we provide evidence, short of a complete 
proof, to show that in general the adaptive algorithm may recover from this 
situation. 

6. The function mapping the identification parameter E to the control pa- 
rameter ©, A, will be specified later in the discussion of the results we are 
about to present. For the time being, the main property of importance is that 

leads to a stable closed-loop system, either via pole placement or via LQ 
design. In order to be able to apply the averaging result we also need the 
mapping to be Lipschitz continuous in a neighborhood of the estimates 
produced by the adaptive algorithm which follows from Assumption 3.1. 



Main Points of Section 

We have described more completely adaptive- Q methods by introducing two 
adaptation algorithms, the classical adaptive algorithm and the filtered excitation 
algorithm. The control system we are about to analyze consists of (2.7) together 
with either (3.1) or (3.2). 



7.4 Adaptive Algorithm Analysis: Ideal case 

Let us consider the adaptive algorithm consisting of (2.7) and (3.1) or (3.2) under 
the condition that C a = 0. To fix the ideas we focus on the pole placement prob- 
lem, but it will transpire that the analysis and main results apply to any reasonable 
design rule. First we study the possible steady state behavior of the closed-loop 
system under the condition that the external disturbances are zero (wk = 0) and 
that the probing dk signal is stationary and sufficiently rich. The steady state anal- 
ysis indicates that the algorithm may be successful in an adaptive context. Next 
we consider an averaging analysis, first without signal perturbation, next with sig- 
nal perturbation. The results provide a fairly complete picture of the cases where 
the algorithm can be used with success and provide pointers to how it may fail. 
This topic is taken up in the next section, where we study the influence of plants 
not belonging to the assumed model class. 
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Steady State Analysis 

Let us consider the situation w i,* = 0, W2,k = 0 and Zf,k — z f,k = 0. The latter 
condition implies for either algorithm (3.1) or (3.2) that = H and (©*, A© = 
(E) = (©, A). There is no adaptation. Obviously, we want the closed-loop 
system to behave well under this condition for as there is no adaptation, there is 
no way the control algorithm can improve the performance. It can be argued that 
this property is almost necessary for any adaptive algorithm. In the literature it is 
often referred to as tunability or the tuning property, see Mareels and Polderman 
(1996). We show that either algorithm, the classic adaptive algorithm (3.2) as well 
as the filtered excitation (3.1), possesses the tuning property. 

Indeed, given z/,k — z f,k = 0, we describe the adaptive closed-loop system 
via the following time-invariant system, making use of the fact that w\^ = 0 and 
W2,k = 0. 



Zf,k + 1 
Z*+ 1 

Jf,k+ 1. 





(4.1) 



Because of the stability of A + H C we have that Jc* converges to zero exponen- 
tially, and therefore, this state is omitted from (4.1). Moreover, xk is also omitted, 
as its stability is determined by the stability of the system (4.1). If the above sys- 
tem is stable, so is the complete system. We now exploit the fact zf,k = z f,k to 
rewrite (4. 1) as: 



z fM 1 




A s 0 

-B f E*C s A f 


B S G 0 0 ' 

0 0 0 


z f,k 




■ Bs 
0 




Zk+ 1 


= 


0 0 


Aq + A 0 BqCf 


Zk 


+ 


0 


d k 


%, i 




0 0 


B s © A s 0 


h 




B s 




- Z/,t+l J 




0 0 


0 —BfEC s Af _ 


- Zf.k - 




_ 0 _ 





Observe now that the block diagonal matrix 

r oi 

l~B f Z*C s A f j 

is stable by construction. Moreover, the matrix 

~A q + A 0 B q Cf 
B s & A s 0 

0 —B f aC s A f _ 



(4.2) 



(4.3) 



(4.4) 



is stable by virtue of the design with (©, A) = (E). It follows thus from equa- 

tion (4.2) that zk, Vk, z /,*, Vk an d z f,k are all bounded. More importantly, from 
the observable signals point of view, it appears that the control objective has been 
achieved for the closed-loop system (2.7) with (3.1) or (3.2). Indeed, the closed- 
loop stability and performance hinges on the eigenvalues of the matrices: 
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• A s , the unmodeled dynamics, which are outside the control bandwidth 

• Ay, the filter, free for the designer to choose, as long as it is stable 

• A + HC, the observer eigenvalues for the nominal control design 

• A + BF, the controlled nominal plant, and 

• (4.4), the controlled model for the plant-model mismatch system, which are 
the poles of the closed loop (Q, S). 



The above observation is independent of the nature of dk, and would even be 
true for dk = 0. However, if dk is sufficiently rich in that it satisfies a condition 
like Assumption 6.4.2, then we have the additional result that the only steady 
state parameters are the true system parameters, that is, E = E* and (0, A) = 
(E*) = (©*, A*). Actually, we desire that the spectrum of dk contains at least as 
many distinct frequency lines as there are parameters in E to identify. This follows 
from the following construction. Introduce dk = Vk — Vk and lf,k = z f.k — z f,k- 
Then from (4.1) we have: 



h+i = A s v k 



Zf,k + 1 = A fZf,k ~ B t {a* - a)C s v k - B f a*v k . 



Also, 



( 


~A q + A 0 B q cf 


1 


"o" 


k- 


B s @ A s 0 




If 


V 


0 -B q uC s A f 


) 


_ 0 _ 



from which it follows that Zf,k = 0 can only occur when E* m E. 
We summarize our observations as follows: 



Theorem 4.1 (Tuning Property). Consider the adaptive systems described by 
either (2.7) with the algorithm (3.1) or adaptive algorithm (3.2). Let the external 
disturbances be zero ( Wk = 0). When the tuning error z.f,k — z f.k is identically 
zero, the algorithm ’s stationary points (E^ = E , (©* , A/.) = ( E)) are such that 

the closed-loop system is stable and the desired control objective is realized. 

Theorem 4.2 (Tuning property with excitation). Consider the adaptive sys- 
tems described by either (2.7) with algorithm (3.1) or (3.2). Let the external dis- 
turbances be zero ( Wk = 0). Let the probing signal be sufficiently rich in that the 
spectrum of dk contains as many distinct frequency lines as there are elements in 
E. The algorithm’s stationary point is unique. E^ = E* and the desired control 
objective is realized. 

Remark. The difference between Theorem 4.1 and Theorem 4.2 is significant. 
Polderman (1989) shows that only in the case of pole placement one can conclu- 
sively infer from Theorem 4.1 that the control achieved in the adaptive algorithm 
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equals the control one would have implemented if the system were completely 
known. In the case of LQ control, the achieved LQ performance is only optimal 
for the model, that is optimal for S, not for the plant E*. Due to lack of excita- 
tion we are unable to observe this in the adaptively controlled loop. However, in 
the presence of excitation, due to the correct identification of E* asymptotically 
optimal performance is obtained, for any control design. This goes a long way in 
convincing us why excitation, via the probing signal dk, is indeed important in an 
adaptive context. It is one of the main motivations for preferring the filtered exci- 
tation algorithm above the classical algorithm. Further motivation will emerge in 
the subsequent analysis. 



Transient Analysis: Ideal Case 

Exploiting standard results in adaptive control one can show that the classical 
algorithm (Algorithm 2), under the assumptions 

• the plant belongs to the model class (Ca = 0) 

• the external disturbances are zero ( Wk = 0) 

• along the solutions of the adaptive algorithm is well defined 

indeed realizes the desired control objective in the limit. Moreover, if the prob- 
ing signal is sufficiently rich, the actual plant will be correctly identified. The 
interested reader is referred to Mareels and Polderman (1996), Chapter 4, for a 
complete proof. 

From a control performance perspective, which is the topic of this book of 
course, this result is however not very informative. Indeed the classical adaptive 
control results do not make any statements about important questions such as: 

• How long do the transients take? 

• How large is a bounded signal? 

Indeed, a moment of reflection indicates that a general result can not make any 
statements about problems of the above nature. A result valid for (almost) all pos- 
sible initial conditions must allow for completely destabilizing controllers. For 
such cases it is not possible to limit either the size of the signals encountered in 
a transient nor the time it takes to reach the asymptotic performance. In general 
this situation is aggravated by imposing the condition of slow adaptation. How- 
ever, in the present situation, we can avoid the above disappointments, because 
we start from the premise that the nominal controller, however unsatisfactory its 
performance, is capable of stabilizing the actual plant. We proceed therefore with 
Algorithm 1 (Filtered excitation), exploiting explicitly the fact that our initial in- 
formation suffices to stabilize the system. By injecting a sufficiently rich prob- 
ing signal, which is conveniently filtered, we show that the adaptation improves 
(slowly) our information about the actual plant to be controlled, hence improving 
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our ability to control it. We regard this control strategy as one where we exploit 
the robustness margin of a robust stabilizing controller to such an extent that we 
learn the plant to be controlled in such a way as to improve the control perfor- 
mance. The existence of a robustness margin is crucial throughout the adaptation 
process. The more robust the controller, it turns out, the easier the adaptation pro- 
cess becomes. The algorithm is clearly achieving a successful symbiosis of robust 
control and adaptive control. Averaging techniques are exploited to establish the 
above results. 

Let us be explicit about our stabilization premise: 

Hypothesis 4.3. Along the sequence of estimates 3^, k = 0, 1, . . ., the design 
rule is such that (0&, A/.) = (3^) is a stabilizing controller for the actual 

plant to be controlled. 

In the ideal scenario, the validity of this hypothesis is based on the following 
observations. 

Introduce 5* = Vk - h, lf,k = $f,k ~ if.k and a k = a k - 3*. Along the 
solutions of the adaptive algorithms we have then, up to terms in Wk'. 








By construction we have that the matrices 

U, :] 



A q + Ak 0 BqC f 
B s @ k A s 0 

0 -B f a k C s A s 



(4.5) 



(4.6) 



(4.7) 



are stable (in that the eigenvalues for each instant k are less than 1 in modulus). 
Hence, provided 3* is slowly time varying in that || 3*+i — 3*|| is sufficiently 
small and 3* is sufficiently small the overall system will be stable. As we will 
show | 3* | is monotonically nonincreasing, and || 3^ + i — 3* || is governed by pt. 

Hence, assuming || 3* |J is sufficiently small, and /x is sufficiently small, we 
have that Hypothesis 4.3 is satisfied along the solutions of the adaptive system. 

In order to see that || 3* || is decreasing, we proceed as follows. Introduce, 



A,yjtt i = AfXij'k - BfEijC s hk, 

hk+\ = A s hk + B s ®kZk- 



(4.8) 

(4.9) 
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We have then, comparing (2.7) with (4.9) under the condition that C\ = 0, that 

Zf,k = Y yij-k S *j + Y X U,k Sy , (4.10) 

ij U 

and also 

Y Vi* s 0\* + Y S *M + O(n). (4.11) 

ij ij 

Here 0{p) stands for a term fk that can be over bounded as \fk\ < K \ /x for 
some Ki > 0 (see also Appendix C). Moreover, because the filter ( Af,Bf,Cf ) 
is designed such that it eliminates the spectrum of dk, and because we assume that 
the driving signals Wk are orthogonal to dk, it follows that, for constant 0: 

, N + ™ 

Jim — Yij,k^ij,k-m = 0 for all m =0, 1,2,... and all i, j. 

^ k=m + 1 

The above expression embodies the most important design difficulty, on the one 
side dk must be sufficiently rich to lead to the identification of S*, but we also 
need to filter it out of the Q loop via ( Af,Bf,Cf ), complicating the controller 
design. 

For the adaptive update equation, see (3.1), we find thus after substituting (4.10) 
and (4.11): 

Eyjfl = Bij,k ~ IkYij,k Y, (ytt’k + ^et,k) (3j r — + 0{p?) 

for all i,j;k. (4.12) 

This equation (4.12) is in the standard form to apply averaging theory, see Ap- 
pendix C. Using the results from Appendix C we find for the averaged equation 

vec(Sj” 1 ) =vec(Ef) -^rvec(3f - S*) (4.13) 

where the matrix T contains as elements 
1 N 

/foo N^j' kYit ’ k 

in appropriate order. For sufficiently rich dk, the matrix T is positive definite, 
T = T' > 0. It follows that S" 1 ’ converges exponentially to H*, for sufficiently 
small p and for sufficiently rich dk . Theorem C.4.2 informs us that, 




Because dk has a finite spectrum, we are able to obtain the stronger result 
| E k ~ ^“k II = 0{il), for all k. 

Hence for sufficiently small /x, optimal control is achieved up to small errors 
of the order of the adaptation step size, without losing the stability property of the 
initial stabilizing controller. We summarize: 
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Theorem 4.4 (Ideal case, filtered excitation). Consider the adaptive system 
(2.7) with (3.1) under the condition that the plant belongs to the model class 
Ca = 0. Let Assumption 3.1 be satisfied. Let d have a finite spectrum, but suf- 
ficiently rich as to enforce unique identifiability of E* (d’s spectrum contains as 
many distinct frequency lines as there are elements in E*). Let ( Ay , B f, C f) be 
chosen as to null the spectrum of d (2.5). Let Wk be uncorrelated with dk. Then 
there exists a p* > 0 , such that for all p e ( 0 , p*) and for all initial conditions 
such that Eo leads to a stable closed loop, we have that: 

1. The adaptive system state is bounded, more precisely there exists constants 
Ci > C 2 , C 3 > 0 and 0 < k < 1 such that 

\\X k \\<Ci\ k UX 0 II+C 2 M+C 3 IMI, 
x k = (v' k ,v' k ,z' ftk ,Zk,x' k ,x' k ^ , 

and 

2. Exponentially fast optimal control performance is achieved, in that there 
exists positive constants C 4 , C 5 , Cf, > 0 independent of p, with 0 < 1 — 
p*Cn < 1 , such that: 

1% -®?||^'C6(1 - C4/l) k jSo - 3*;| + Csp. 

Remark. 

The above result is applicable to Problems 1 and 2 of Section 7.2. We stress that it 
is important that ( E) is Lipschitz continuous in E, otherwise the averaging result 
is not valid. (See Theorem C.2.2). This Lipschitz continuity may be guaranteed 
via Assumption 3.1. Indeed, under Assumption 3.1, defined via either pole- 
placement or LQ control can be constructed to be Lipschitz continuous along the 
estimates generated by the adaptive algorithm. 

More importantly, the same result would also apply to the whole class of adaptive 
algorithms (&k, A k ) = k (EQ, such that Hypothesis 4.3 is satisfied. For pole 
placement and LQ control, this has been verified, but a range of alternatives is 
conceivable. Most noteworthy is the earlier suggestion to have (EQ = 0 for 
all A: = 0, 1 , kp, that is, wait to update the controller until E * 0 is such that 
controller design via LQ or pole placement does not destroy the robustness mar- 
gin of the initial controller K. Also, k could reflect our desire to improve the 
closed-loop control performance as time progresses by, for example, increasing 
the bandwidth of the controlled loop. 

Finally, we point out how in the case of LQ control may be constructed. We 
consider the LQ index (see Problem 2 of Section 7.2): 

1 N 

J ( ) = - J 2 ( s k Sk + r k rk ) : = ' > 0 



(4.14) 
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To construct let us assume that the matrix pair (2.9) is stabilizable and the 
matrix pair (2.10) is detectable. Denote 

/a, 0 \ 


= 1 


y~Bf ECj Af)’ 


(4.15) 


= 1 


(B S \ 


(4.16) 


= ( 


0 B q Cf). 


(4.17) 


Under the stated conditions, ( , 


) stabilizable and ( , ) detectable, 


we can 


solve the following Riccati equations for unique positive definite and 




= ' + / - 
and 


-( r )(c '+iy l ' 


(4.18) 


= ' + - 


( ' M ' + r 1 ' ■ 


(4.19) 



Here I and are the weighting matrices from the LQ index (4.14). 

Given and we construct 

= ( ')(C ' + R)~ l (4.20) 

and 

= ( ' +S)~ l { ' ), (4.21) 

which have, respectively, the property that — and — are stable matri- 
ces. The controller that solves the optimization of the index J of (4.14) can then 
be implemented in the standard way (see Chapter 2), with observer gain and 
feedback gain . In particular, we have 




and 

© = - . (4.23) 

Because is an affine function of E it follows that , and hence also , , and 
A, © depend on E. Moreover it can be demonstrated that on any open subset of 
detectable matrix pairs ( , (E)) and on any open subset of stabilizable matrix 

pairs ( (E) , ) this dependency on E is analytic. (See Mareels and Polderman 

(1996) for a proof of the analyticity property.) 
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Main Points of Section 

The behavior of the adaptive closed-loop system is studied in the situation that 
the parameterized class of models contains the particular plant, the so-called ideal 
situation. First attention is paid to the possible no-adaptation behaviors. The in- 
direct adaptive algorithms introduced involving either filtered excitation or using 
classical adaptation, both enjoy the tuning property. The tuning property indicates 
that under conditions that the identification error is zero, the adaptively controlled 
closed loop appears to be controlled as if the true system is known. This estab- 
lishes that the steady state behavior of the controlled loop is as good as we can 
hope for. Next we consider the transient behavior. We establish that the filtered 
excitation adaptive algorithm in conjunction with a sufficiently rich probing sig- 
nal is capable of identifying the plant and hence achieves near optimal control 
performance. The result holds under the condition of sufficiently slow adapta- 
tion. The key idea in the filtered excitation algorithm is that as far as the probing 
signal is concerned the loop is essentially open. This achieves unbiased identifi- 
cation and allows one to exploit the robustness margin of the stabilizing controller 
without compromising the performance. Robust control and adaptation are truly 
complementary. 



7.5 Adaptive Algorithm Analysis: Nonideal Case 

The most important realization is that in our set up the tuning property remains 
valid in the presence of unmodeled dynamics for either the classical or the filtered 
excitation adaptive algorithm. This is a consequence of the fact that we start from 
the premise that our initial controller is stabilizing. It is in sharp contrast with 
the classical position in adaptive control, where arbitrarily small perturbations 
may lead to instability, destroying the tuning property. Moreover, if we restrict 
ourselves to the filtered excitation algorithm, we will deduce that in spite of mod- 
eling errors, we do identify the best possible model in our model class to represent 
S. This property does not hold for the classic adaptive algorithm due to the highly 
nonlinear interaction between identification and control. As a consequence the 
filtered excitation algorithm has a much larger robustness margin for inadequate 
modeling of the S parameter than the classic algorithm. The main result is there- 
fore restricted to the filtered excitation algorithm. 

Tuning Property 

Although we argue that the tuning property is close to necessary for an adaptive 
algorithm, it is clear that in the nonideal case this property plays a less important 
role. This is because the condition zf,k ~ z f,k = 0 can hardly be expected to be 
satisfied along the solutions of the adaptive system. Following the same arguments 
as in the ideal case we find nevertheless that the tuning property also holds in the 
nonideal case. Indeed, the stability of the closed-loop adaptive system hinges on 
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the stability of the system (neglecting the driving signals Wk)'. 




Again using zf,k = z f,k, it follows that the stability of the loop depends only on 
the stability of the matrix 



( A a 0 0 \ 

0 A s 0 

-B f C A ~B f E*C s A f ) 

and the design matrix 

( Aq + A 0 BqCf\ 

B s & A s 0 . 

0 -B q ZC s A f ) 

The former is stable by assumption, the latter by construction. This establishes 
the tuning property. 

Expression (5.1), however makes it very clear that due to the presence of v A 
and certainly in the case of a sufficiently rich dk, we can not expect to have z f,k = 
z/,k- In the absence of any probing signal dk and with no external disturbances 
Wk = 0, it is possible to have zf,k = i f,k- 

Identification Behavior 

Let us now focus on what model will be identified in closed loop via the filtered 
excitation adaptive algorithm. Again, we rely on slow adaptation and the stabil- 
ity Hypothesis 4.3. Whereas in the ideal case, it is clear that Hypothesis 4.3 is 
fulfilled under very reasonable assumptions, this is no longer guaranteed in the 
nonideal model case. We postpone a discussion of this crucial hypothesis until we 
have a clearer picture of what the possible stationary points for the identification 
algorithm are. 

Clearly, as before we have (see (4.1 1)) 

Zf,k ~^Yij,k^ij,k + ^2^ij,k^ij,k + o in)- 

ij ij 



However 




7.5. Adaptive Algorithm Analysis: Nonideal Case 197 



where 

hi.* 1 1 = A/vi,* - B f C A v u , 
v\,k+\ — A a v \ ,* + B A dk, 

V2,k+\ — AfV2,k ~ BfC A V2,k, 

V2, k+\ = A A V2,k + B A @kZk- 

The adaptive update equation can thus be written as: 
a ij,k+l 

= E ijt k - {Yti,k + hi.k) ( E u,k - ^a) + (vi.it + v 2 ,k)j + 0 {p}). 

(5.2) 

In the above expressions A and V 2 ,* are functions of E*, but as observed before, 
for fixed E we clearly have 



l N 

*= i 

because both V2 and X t t are filtered versions of the signal z which does not contain 
the spectrum of d , as this is eliminated by the filter ( Af , Bf,C /). Computing the 
averaged equation for (5.2), we have thus: 

vec(S^ 1 ) = vec(3%°) k - /rT vec (E™ - E*) +gM, 

where M consists of the elements 




in appropriate order. If follows that the averaged equation, under persistency of 
excitation such that I" = T' > 0, has a unique equilibrium: 

vec (E™) = vec (E*) + F _1 M. 

Reinterpreting the above averages in the frequency domain, we see that the iden- 
tification process is equivalent to finding the best E parameter in an i 2 approxi- 
mation sense, that is 



E^ = argmin ||(S (z) - EB(z))d\\ 2 , 
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where 



S(z) = C A ( zl - A a ) 1 B a + a*B ( z ) . 

This is clearly the best we can achieve in the present setting, but unfortunately it 
is not directly helpful in a control setting. We discuss this in the next subsection. 
Let us now summarize our main result thus far: 

Theorem 5.1. Consider the adaptive system (2.7) together with (3.1). Let As- 
sumption 3.1 and Hypothesis 4.3 hold. Assume that the external probing signal d 
is sufficiently exciting in that US (z) — SB (z) c/|| 2 has a unique minimizer. More- 
over the filter (Ay, By, C y) nulls the signal d. Assume that the external signal w 
is stationary and has a spectrum which does not overlap with that of d in that: 

1 N 

lim — V* Wkdk = 0. 
k—>oo N “ 

Then for all initial conditions satisfying Hypothesis 4.3 in a compact domain there 
exists a p* > 0 such that for all p e (0, p*) the adaptively controlled loop is 
stable, in that all signals remain bounded. Moreover, 

1. =8(p\ 

2. limsup | Si — 3^ || = <5(/t), 



for some order function S (p.) such that lim /t ^o (5(/i) = 0. 

The main difficulty is of course Hypothesis 4.3 which we discuss now. 

Identification for Control 

As indicated earlier, the asymptotic performance of the adaptive algorithm is gov- 
erned by: 

Hoc = argmin ||(S(z) - SB(z))d\\ 2 . 

The corresponding closed-loop stability depends on 
(Boc.Aoc)^ (Boo), 

where by construction ensures that the matrix 

( A s 0 B s @ oo \ 

-ByS^Cs Ay 0 

0 BqCf Aq + Aooy 



(5.3) 
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is a stable matrix. The closed-loop stability of the system is however determined 
by the stability properties of the matrix 

f A a 0 0 

0 A s 0 

— BfC A — Bfa*C s Af 

K 0 0 B q Cf 

Let us interpret these in terms of transfer functions. Introduce 
Soo(z) = Soo B(z), 

S(z ) = C A G zl ~ A a ) _1 B a + E B(z), 

Q(z) = ©oo (zl - A, - Aoo) -1 B q C f ( zl - A/) -1 B f , 

A (z) = S(z) - Soo(z). 

Thus (5.3) being a stable matrix states that (5oo(z)> Q(z)) is a stable loop, and 
(5.4) stable expresses that (S(z), Q(z.)) is a stabilizing pair. A sufficient condition 
for the latter is that: 

* ||- 2(z)5oo(z))“ 1 A(z|t < 1. (5.5) 

Via the adaptive algorithm we have ensured that 

II A(zM||l < 1- (5-6) 

which does go a long way in establishing (5.5) but is not quite enough. It is in- 
deed possible that the minimization of (5.6) does not yield (5.5), and may even 
lead to instability in the closed-loop adaptive system. This indicates that the adap- 
tive algorithm leads to unacceptable behavior. A finite-time averaging result (see 
Appendix C), allows us to conclude that the adaptive algorithm will indeed try 
to identify S^iz). This leads to a temporarily unstable closed loop, characterized 
by exploding signals. At this point averaging would no longer be valid. Hypothe- 
sis 4.3 being violated. But it does indicate that large signals in the loop are to be 
expected. Invariably the performance of such a controlled system is bad, even if 
the adaptive loop may recover from this explosive situation. Understanding what 
type of behavior ensues from this is nontrivial. For a discussion of the difficul- 
ties one may encounter we refer to Mareels and Polderman (1996, Chapter 9). 
Suffice it to say that chaotic dynamics and instability phenomena belong to the 
possibilities. 

In order to have that the minimization of (5.6) leads to (5.5) being satisfied, we 
should have either 

1. a sufficiently general model to ensure that Ca will be small, or 

2. ensure that outside the frequency spectrum of d the controlled loop 
GSoo(z), Q(z)) has small gain. 



B A ©00 
B s & oo 




200 Chapter 7. Indirect (Q, S) Adaptive Control 



The link between identification and control is obvious in the equations (5.5) 
and (5.6) and has been the focus of much research. See, for example Partanen 
(1995), Lee (1994), Gevers (1993). 

Main Points of Section 

In the nonideal case, the model class is insufficient to describe the mismatch be- 
tween the plant and the nominal plant; the filtered excitation adaptive algorithm 
attempts to identify the best possible model in an £2 sense. Unfortunately, this may 
not be enough to guarantee stability let alone performance. Indeed despite the fact 
that the initial model and controller is stable it may be that the best possible model 
in the model class leads to instability. The interdependency of identification and 
control is clearly identified. The key design variables are the choice of model 
class, the probing signal and the control objective, as exhibited in equations (5.6) 
and (5.5). 

Example. First we demonstrate the idea behind the filtered excitation adaptive al- 
gorithm, without using the ( Q , S) framework. The example will deviate slightly 
from the theory developed in this chapter in order to illustrate the flexibility of- 
fered by the averaging analysis. The example is an abstraction of a problem en- 
countered in the control of the profile of rolled steel products. 

Consider the plant represented in Figure 5.1. The input u and output e are mea- 
surable. The control objective is to regulate e to zero as fast as possible. The 
signal w is an unknown constant. The plant output y is not measurable. The gain 
g £ (0, g), is unknown but g is a known constant. 

The proposed solution, in the light of the filtered excitation adaptive algorithm, 
is to use a probing signal dk = (— 1 ) k d, where d is a small constant, leading to 
an acceptable error in the regulation objective. The controlled plant becomes as 
presented in Figure 5.2. 

When g = g, then we have dead beat response and e is regulated in three time 
steps. More precisely, with q~ l the unit delay operator 

q 3 e = (q - 1 ){q - \){q + |)th + g(q - \){q + j)(-l ) k d. 

The probing signal leads thus to a steady state error of \gd/2\ in magnitude. Of 
course, due to the integral in the plant, there is no steady state error for any con- 
stant U). 




FIGURE 5.1. Plant 
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FIGURE 5.2. Controlled loop 

It is easily verified that the above system is stable for all g/g e (0, 2). It is thus 
advantageous to over estimate g. The filtered excitation adaptive algorithm can be 
implemented as follows: 

gk+i = 8k ~ (e k + ^(-1)^ ; 80 = 8- 

The complete control loop is illustrated in Figure 5.3. 

Indeed because of the filter we expect the steady state behavior of ek due to 
the probing signal to be —(gd/2)(—l) k , our estimate for this is — (gkd/ 2)(—\) k , 
which leads to the above update law. Now provided g/gk £ (0, 2) and for suf- 
ficiently small pt, we can look at the averaged update equation to see how the 
adaptive system is going to respond. According to our development this leads to 

~av ft av ( sd 8k V d\ ~av - 

8k+i = s k + 80 =go = g ’ 

or 



8k+i 8k 2 fet g ) ' 




FIGURE 5.3. Adaptive control loop 
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FIGURE 5.4. Response of g 




Time index 



FIGURE 5.5. Response of e 

Hence, as expected g% v converges monotonically to g whenever 0 < /id < 2. 
Now from Theorem 4.4 we conclude that for all g/go e (0, 2) 

Isk-sifeoui) for all k. 

This leads to asymptotically near optimal performance for all g e (0, g), actually 
for all g e (0, 2 g — e), where s is any small constant 1 » e > p. > 0. 

A response of the algorithm is illustrated in Figures 5.4 and 5.5. Figure 5.4 
illustrates the response of the g variable, while Figure 5.5 displays the response 
of the regulated variable. For the simulation we choose ptd = 0.2, and all other 
initial conditions set to 1 . Notice that the averaging approximation predicts the 
closed-loop behavior extremely well. 

As can be seen from this example, stability of the plant to be controlled is not 
essential for the filtered excitation algorithm to be applied. The stability of the 
closed loop is, of course, important. 

Simulation Results 

In this subsection, we present simulations for the case where an explicit adaptive 
LQG algorithm is used to design an adaptive Q filter, denoted Qk. The actual 
plant G(S), and the nominal controller K used are designed based on the nominal 
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plant G of the example in Section 5.2. The following LQ index penalizing r and 
s is used in the design of the adaptive LQG augmentation Q%. 

7 L g=hm £>?+*?). (5.7) 



Table 5.1 shows a performance index comparison for the following various 
cases. The first is where the actual plant G(S ) is controlled by the LQG controller 
K for the nominal plant G with no adaptive augmentation. The second case is 
when the LQG controller for G is augmented with an indirect LQG adaptive- Q 
algorithm. A third order model S is assumed in this case, and the estimate of S 
‘converges’ to 



0.381z -1 — 0.0925z -2 — 0.358 8z -3 
~~ 1 - 0.423 9z-‘ - 0.439 2z“ 2 -0.018 lz- 3 ' n ' M 

A marked improvement over that of the nonadaptive case is recorded. Note that 
the average is taken after the identification algorithm ‘converges’ . The third case 
is for the actual plant, G(S), controlled by the corresponding LQG controller, 
designed based on knowledge of G(S ) rather than on that of a nominal model G. 
Clearly, the performance of the adaptive scheme (Case 2) is drastically better than 
for the nonadaptive case, and approaches that of the optimal scheme (Case 3), 
confirming the performance enhancement ability of the technique. 



Case 


+ 0 - 005 "*) 


1. Actual plant G{S) with LQG 
controller for nominal plant G. 


0.423 0 


2. Actual plant G(S ) with nominal 
LQG controller and adaptive Qk . 


0.1860 


3. Actual plant G(S) with op- 
timal LQG controller for G(S). 


0.1756 



TABLE 5.1. Comparison of performance 

In a second simulation run, we work with a plant G which is not stabilized by 
the nominal controller K. Again S is identified on line using a third order model 
and there is employed an adaptive LQG algorithm, as in the run above, to design a 
Qk to augment K. Figure 5.6 shows the plant output and input. In this instance, the 
adaptive augmentation, Qk together with K stabilizes the plant. The results show 
that the technique not only enhances performance but also can achieve robustness 
enhancement. 
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Main Points of Section 

The first example serves to illustrate the powerful nature of the averaging tech- 
niques. Clearly the analysis can be used for design purposes. The filtered exci- 
tation algorithm provides a suitable way of injecting an external signal such as 
to identify the plant characteristics in a particular frequency range, without com- 
promising the control performance too much. The trade off between desired con- 
trol objective and the identification requirements can easily be analyzed in the 
frequency domain. The second example clearly demonstrates the strength of the 
(Q, S) framework. 



7.6 Notes and References 

Indirect adaptive control has a played an important role in the development of 
control theory. It has been treated extensively in books such as Goodwin and Sin 
(1984), Sastry and Bodson (1989), Narendra and Annaswamy (1989), Mareels 
and Polderman (1996). Our premise here is of course the availability of a sta- 
bilizing, not necessarily well performing controller. This leads to a significant 
departure of the usual approach to the design of adaptive systems. It imposes a 
lot of structure on the adaptive problem which can be exploited to advantage. The 
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developments of these ideas can be traced to Tay (1989) and Wang (1991). In or- 
der to achieve identification in closed loop without compromising the control too 
much, we introduced the filtered excitation adaptive scheme. This has not been 
treated before in the literature and much of the presentation here is new. The aver- 
aging ideas which are crucial to the analysis have been developed in, e.g. Mareels 
and Polderman (1996) and Solo and Kong (1995). Averaging has a long history in 
adaptive control and signal processing analysis, see for example Anderson et al. 
(1986). 

Problems 

1. Reconsider the example in the text in the (Q. S) framework. Let G = 
l/(z — 1) with N = l/z and M — (z — l)/z. Let K — 1 with U = V = 
1. Show that S = (z — \)(g — l)/[z(z + (g — 1))]. Use as parameterized 
function class B(z, E) = ((z — l)/z)(E/(z + E)) with E e (—1, 1). Use 
the same probing signal as in the text d(k ) = (— 1 ) k d. Consider the plug 
in controller to take the form Q(z ) = q\ (z + l)(z + qi)/{z 2 + qiz + qf). 
Achieve dead beat control for the (Q, S) loop, that is, compute . Show 
that is Lipschitz continuous on the domain E e (0, 2). Show that for 
correctly tuned E this strategy achieves dead beat control for the complete 
closed loop. Now implement the filtered excitation algorithm. 

Remark. Observe that this control strategy is more complicated than the 
direct implementation discussed in the text. The advantage of the ( Q , .S') 
framework is that it leads to a more robust control loop with respect to 
other disturbances. 

2. Using the example in Section 7.5, explore the effect of using other probing 
signals of the form d(k) = d cos (cok) with appropriate filter in the Q- loop. 
Show that as u> becomes smaller it becomes increasingly difficult to achieve 
good regulation behavior. Why is this? 

3. The adaptive control algorithm, like all indirect adaptive control algorithms 
has no global stability property. In particular, as indicated, the filtered ex- 
citation algorithm may fail when the Hypothesis 4.3 fails. This may be il- 
lustrated using the example in Section 7.5 by taking initial conditions as, 
for example, go = 0, y — 10. The stability hypothesis fails and limit cycle 
behavior is observed. What happens when go g? 




CHAPTER 8 



Adaptive - Q Application to 
Nonlinear Systems 

8.1 Introduction 

For nonlinear plants there has evolved both open-loop and closed-loop optimal 
control theory. Optimal feedback control for very general plants and indices is 
very new and appealing, see Elliott et al. (1994), but requires infinite dimen- 
sional controllers designed using off-line infinite dimensional calculations. Op- 
timal open-loop nonlinear control methods are considered very elegant in theory, 
but lack robustness in practice, see Sage and White (1977). Can adaptive- <2 meth- 
ods somehow bridge the gap between these elegant theories and result in practical 
feedback controllers that have most of the benefits of both approaches without the 
disadvantages of each? We proceed as in the work of Imae, Irlicht, Obinata and 
Moore (1992) and Irlicht and Moore (1991). 

In the optimal open-loop control approach to nonlinear control, a nonlinear 
mathematical model of the process is first formulated based on the fundamental 
laws in operation or via identification techniques. Next, a performance index is 
derived which reflects the various cost factors associated with the implementation 
of any control signal. Then, off-line calculations lead to an optimal control law u* 
via one of the various methods of optimal control, see for example Teo, Goh and 
Wong (1991). In theory then, applying such a control law to the physical process 
should result in optimal performance. However, the process is rarely modeled 
accurately, and frequently is subject to stochastic disturbances. Consequently, the 
application of the “optimal” control signal u* results in poor performance, in that 
the process output y differs from y*, the output of the idealized process model. 

A standard approach to enhance open-loop optimal control performance, see 
for example Anderson and Moore (1989), is to work with the difference between 
the ideal optimal process output trajectory y* and the actual process output y, 
denoted Sy, and the difference 8u, between the optimal control u* for the nom- 
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inal model and any actual control signal u applied. For nominal plants and per- 
formance indices with suitably smooth nonlinearities, a linearization of the pro- 
cess allows an approximate linear time-varying dynamic model for relating 8y 
to 8u. With this model, and an associated quadratic index also derived from the 
Taylor series expansion function, optimal linear quadratic feedback regulator the- 
ory can be applied to calculate 8u in terms of 8y which is measurable, so as 
to regulate 8y to zero, or equivalently to force the actual plant to track closely 
the optimal trajectory for the nominal plant. Robust regulator designs based on 
optimal theory, perhaps via H 0 0 or LQG/LTR, could be expected to lead to per- 
formance improvement over a wider range of perturbations on the nominal plant 
model. 

Even with the application of linearization and feedback regulation to enhance 
optimal control strategies, there can still be problems with external disturbances 
and modeling errors. The linearization itself may be a poor approximation when 
there are large perturbations from the optimal trajectory. 

In this chapter, it is proposed to apply the adaptive- Q techniques developed 
for linear systems so as to achieve high performance in the face of nonlineari- 
ties and uncertainties, that is to assist in regulation of the actual plant so that it 
behaves as closely as possible to the nominal (idealized) model under optimal 
control. Some analysis results are presented giving stability properties of the opti- 
mal/adaptive scheme, and certain relevant nonlinear coprime factorization results 
are summarized. Simulation results demonstrate the effectiveness of the various 
control strategies, and the possibility of further performance enhancement based 
on functional learning is developed. 



8.2 Adaptive - Q Method for Nonlinear Control 

In this section, we first introduce a nonlinear plant model and associated non- 
linear performance index. Next, we perform a linearization, then apply feedback 
regulation to the linearized model to achieve a robust controller. Finally we apply 
the adaptive- Q algorithms to this robust controller. There is a mild generalization 
for dependence of the linear blocks (operators) on the desired trajectory. Since 
these operators are inherently time varying, the notion of time-varying coprime 
factorizations is developed. 



Signal Model, Optimal Index and Linearization 

Consider some nonlinear plant, denoted G, and a generalized nonlinear nominal 
plant model G, an approximation for G: 

G : Xk+ i = f(xk, uk ), yk = h(xk, uO, (2.1) 
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with /(•, •) and h(-. •) e C 1 , the class of continuously differentiable functions. 
Consider also some performance index over the time interval [0, T] 

1 T 

1 (*0> M|0,f|) - - Uk) ’ ( 2 - 2 ) 

1 k= 0 



Assume that (2.2) can be minimized subject to (2.1), and that the associated op- 
timal control is given by u*, the optimal state trajectory is x*, and the optimal 
output trajectory is y* so that in an operator notation y* = Gu*, where G is 
initial condition dependent. For details on such calculations see Teo et al. (1991). 

Consider now a linearized version of the above plant model driven by u* and 
with states x*, denoted AG*: 

AG* : 8xk+ i = A8xk + B8uc, <5xo = 0, 

8y k = C8x k + D8u k . 



In obvious notation, 



( XiJ ,,c lD ) = fVV«) 

\3x 3 u dx du ) ( * 



are time- varying matrices since x* and u* are time dependent. We take the liberty 
here to use the operator notation 



Syk = A G*8uk- 



(2.4) 



The following shorthand notation is a natural extension of the block notation of 
earlier chapters for time-invariant systems to time- varying systems. The asterisk 
highlights the optimal state and control dependence. 



AG* : 



A B 
C D 



(2.5) 



Let us denote AG as the operator of the system with input A u = u — u* and 
output Ay = y — y*. Of course, the ‘linearization’ can only be a good one and 
the following design approach effective if the actual plant is not too different in 
behavior from that of the model G. 

Let us associate with the linearized model a quadratic performance index pe- 
nalizing departures Ay and Am away from the optimal trajectory: 

1 T 

= ( 2 - 6 ) 
k = 0 

where, 




Q* = Q*' > 0, Q* - S*(R*)~ l S* > 0, R* = R*' > 0. 



(2.7) 
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Here e is interpreted as a disturbance response which we seek to minimize in an 
rms sense. Of course, A I* and thus L*, can be generated from a Taylor series 
expansion for I about I* = (1 /T) Ylk=o^( x k’ U P t0 second order term. 
In fact the first two terms are zero due to optimality and the second term can be 
selected as A I* with L*L*' the Hessian matrix of I*. Other selections for AI* 
could be simpler and even more appropriate. 

As already noted, we assume that u*, x*, y* are known a priori from an open- 
loop off-line design such as found in books on nonlinear optimal control, see for 
example Teo et al. (1991). However when u* is applied to an actual plant G, which 
includes unmodeled disturbances and/or dynamics, there are departures from the 
optimal trajectories. With departures Ay = y — y* measured on-line, a standard 
approach is to apply control adjustments A u — u — u* to the optimal control 
by means of output feedback control to minimize (2.6). Thus for the augmented 
plant arrangement, denoted Pa, and depicted in Figure 2.1, let us consider a linear 
feedback regulator. We base such a design on the linearized situation depicted in 
Figure 2.2 where the linearized nominal plant, denoted P, is given from 

p= \* p^]’ — [T} pi2=a g *- (2 - 8) 

The star terms Pn, P21 are not of interest for the subsequent analysis. Of course 
in Figure 2.2 the outputs e and Ay are not identical to those of Figure 2.1, but 
they approximate these. 




FIGURE 2.1. The augmented plant arrangement 




FIGURE 2.2. The linearized augmented plant 
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Feedback Regulator for Linearized Model 

Let the regulator of the linearized model P above, based on the nominal model 
AG* (equation (2.3)), be given by 

K* : 8xk+ i = A8xk + B8uk — Hrk, 8x o = 0, (2.9) 

rk = 8yk — C8xk — D8uk, 8uk = F8xk = K*8yk- 

Here r is the estimator residual, 8x is the estimate of 8x and H and F are time- 
varying matrices formed, perhaps via standard LQG/LTR theory of Chapter 4, 
see also Anderson and Moore (1989), so that under uniform stability of A, B and 
uniform detectability of A, C the following systems are exponentially stable: 

&+l = (A + BF)£ k , S k+ 1 = (A + HQSk- (2.10) 

Actually, the important aspect of the LQG design for our purposes is that un- 
der the relevant uniform stabilizability and uniform detectability assumptions, the 
(time- varying) gains H, F exist, and are given from the solution of two Riccati 
equations with no finite escape time. Moreover, for the limiting case when the 
time horizon T becomes infinite, the controller K* stabilizes AG*. 

It is well known that the LQG controller (2.9) for the linearized plants (2.3), 
although optimal for the nominal linear time- varying plant for the assumed noise 
environment, may be far from optimal in other than the nominal noise environ- 
ments, or in the presence of structured or unstructured perturbations on (2.3). 
Stability may be lost even for small variations from the nominal plant. 

Methods to enhance LQG regulator robustness exist, such as modifying Q c , 
S c , R, (usually S c = 0) selections, or assumed noise environments, as when loop 
recovery is used. Such techniques could well serve to strengthen the robustness 
properties of the optimal/adaptive schemes studied subsequently. 

In order to proceed, we here merely assume the existence of a controller (2.9) 
stabilizing AG*, although our objective is to achieve a controller which also sta- 
bilizes AG, and achieves a low value of the index A I* when applied to AG. 

Coprime Factorizations 

Now it turns out that most of the coprime factorization results of Chapter 2 de- 
veloped in a time-invariant linear systems context have a natural generalization to 
time-varying systems. The essential requirement for these to hold is linearity, not 
time invariance. Thus many of the equations of Chapter 2 still hold with appro- 
priate interpretation of the notation. Notions of system stability, system inverse, 
series and parallel connection of systems all carry over to this time- varying sys- 
tem context in a natural way. We proceed on this basis, and leave the reader to 
check such details by working through a problem at the end of the chapter. Let 
it suffice here to state that the developments proceed most naturally in the state 
space framework developed in Sections 2.4 and 2.5. 

Here, it is convenient to introduce normalized x '-dependent and if -dependent 
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coprime factorizations for AG* and K *, such that 

AG* = NM~ l = M~ l N 
K* = UV~ l = V~ l U 
satisfy the double Bezout identity, 

_ I’M tf\ j* V —if 

~ y J L-* ** 

■t :]■ 

Here the factors N, M, N , V, M, N, U , V are stable and causal operators. Since 
they are x*-, m “-dependent, and thus time-varying system linear operators, they 
are natural generalizations of the linear time-invariant operators (transfer func- 
tions) of earlier chapters. Here the product notation is that of the concatenation of 
systems (i.e. of linear system operators). 

Now using the notation of (2.5), suitable factorizations are readily verified un- 
der (2.10) as, see also Moore and Tay (1989b), 



(2.13) 



’ v -t/1 [m u 

-N M N V 



( 2 . 11 ) 

( 2 . 12 ) 



I’m t/1 
|a yJ- 


A + BF 


B -H 




F 

_ C + DF 


I 0 

—D I 




r? - e i= 

\_N M J 


” A + HC 


—{B + HD) 


H 


F 

C 


I 

-D 


0 

I 



The Class of all Stabilizing Controllers 

The theory of the class of stabilizing linear controllers for linear plants, spelled 
out in Chapter 2 for the time-variant case, is readily generalized to cope with 
time-varying systems. Again, the strength of the results depends on linearity, not 
the time-invariance. The details are left for the reader to verify in a problem at 
the end of the chapter, (see also Imae et al. (1992), Moore and Tay (1989b), Tay 
and Moore (1990)). Thus, the class of all linear, causal stabilizing controllers for 
AG* (the linearized plant model) under (2.10) can be generated, not surprisingly, 
as depicted in Figure 2.3 using a Jk subsystem defined below, and a so-called 
Q parameterization. Here the blocks AG, H, A, B,C, F and Q are time-varying 
linear system operators. Referring also to Figure 2.4, the subsystem Jk is readily 
extracted. 

J K : 8xk+ 1 = (A + BF)8xk + Bsk - Hr k , 

8uk = F8xk + Sk, n = 8yk - C8xk - D8uk, 



(2.15) 
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FIGURE 2.3. Class of all stabilizing controllers — the linear time- varying 




FIGURE 2.4. Class of all stabilizing time-varying linear controllers 
equivalently, 




In the Figure 2.4, Q is arbitrary within the class of all linear, timt 
lusal bounded-input, bounded-output (BIBO) stable operators. Thus: 

K*(Q) = V(Q)V~\Q) = V~\Q)U(Q ), 

U{Q) = U + MQ, V (Q) = V + NQ, 

U(Q) = U + QM, V (Q) = V + QN, 
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or equivalently, after some manipulations involving (2.12) and (2.13), 

K*(Q) = K + V~ 1 Q(I + V~ l NQ)~ l V~ l . (2.18) 

Simple manipulations also give an alternative expression for r, as 

r = M8y - N8u. (2.19) 

It is known that the closed-loop transfer functions (operators) of Figure 2.4 are 
affine in Q, which facilitates either off-line or on-line optimization of such Q 
dependent transfer operators. We proceed with a class of on-line optimizations. 



Adaptive- Q Control 

Our proposal is to implement a controller K*(Q ) for some adaptive Q-scheme 
applied to AG. The intention is for Q to be chosen to ensure that K*(Q ) stabilizes 
the feedback loop and thereby the original plant G, and moreover, achieves good 
performance in terms of the index A I* of (2.6). Thus consider the arrangement 
of Figure 2.5 where the block P is characterized by AG and L 

A refinement on this proposal is to consider a two-degree-of-freedom con- 
troller scheme. This is depicted in Figure 2.6. As discussed in Chapter 2, it can be 
derived from a one-degree-of-freedom controller arrangement for an augmented 
plant = [ g, ], reorganized as a two-degree-of-freedom arrangement for G. The 
objective is to select = [2/ 2] causal, bounded-input, bounded-output opera- 
tors on line so that the response e is minimized in an i 2 sense, see also the work 
of Tay and Moore (1990). 

In order to present a least squares algorithm for selection of Q, generalizing the 
schemes of Chapter 6 to the time- varying case as in the schemes of Moore and 
Tay (1989b), some preprocessing of the signals e, 8u,8y is required. 



Prefiltering 

Using operator notation, we define filtered variables 



H = 



ft] 




f = e — P\iMs. 



( 2 . 20 ) 



Least Squares Selection 

To fix the ideas, and to simplify notation, we assume r and ,v to be scalar sig- 
nals in the subsequent developments. Let us define a (possibly time-varying) 
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single-input, single-output, discrete-time version of in terms of a unit delay 
operator q~ l . 



G/OT 1 ) 

GO? -1 ) 

or 1 ) 



Y + yiq 1 + - + Y P q p 

1 + a\ q~ l -\ 1- a n q~ n ’ 

P + Pi q~ l + ■ • ■ + p m q~ m 
1 + ct\q~ x H 1- a n q~ n 

[G/Oz -1 ) GO? -1 )] ’ 



( 2 . 21 ) 



with (possibly time- varying) parameters a,- , Pi , Yi • The following state (regres- 
sion) vector in discrete time is 



= [-■**- 1 ••• Sk-n r k r k—m 0) k 0>k-p\- ( 2 - 22 ) 

The dimensions n , m , p are set from an implementation convenience/performance 
trade-off. In the adaptive- Q case, the parameters are time- varying result- 
ing from least squares calculations given below. We assume a unit delay 
in calculations. Thus 9 is replaced by Ok - 1 and the filter with operator 
k = [ Qfk Qk ] is implemented with parameters (time-varying in general) 



0k~ Ol n k Pok ■ ■ ■ Pmk YOk ■ 



We seek selections of Ok so that the adaptive controller minimizes the norm of 
the response e*. With suitable initializing we have the adaptive- Q arrangement of 
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FIGURE 2.8. Two degree-of-freedom adaptive- Q scheme 
Figure 2.6 with equations 

Ok = Ok - 1 + Pk<t>kh/k- u 
h/k-l = ft — 4>k0k- 1, 
ek/k = ft “ Viflk, 

Pk = fy = Pk-1 - Pk-l4>kU + 4>kPk-l$k)~ l 4>kPk-l, ^ ^ 

4>k = [ (h-l/k-l ~ ft- 1) • • • ( h-n/k-n ~ $k-n) 

-%2,k ■■■ —%2,k-m ~%l,k ••• ~ ?1 ,k-m ]• 

The complete adaptive- Q scheme is a combination of Figures 2.6 and 2.7 with key 
equations (2.14) and (2.24), see also Figure 2.8. A number of remarks concerning 
the algorithm are now in order. 

The algorithms (2.24) should be modified to ensure that Ok is projected into 
a restricted domain, such as || k\\ < G for some norm and some fixed e. Such 
projections can be guided by the theory discussed in the next section. 

To achieve convergence of Ok, then Pk must approach zero, or equivalently, (pk 
must be persistently exciting in some sense. However, parameter convergence is 
not strictly necessary to achieve performance enhancement. With more general 
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algorithms which involve resetting or forgetting, then care must be taken to avoid 
ill-conditioning of Pk, as can occur when there is instability. 

It turns out that appropriate scaling can be crucial to achieve the best possible 
performance enhancement. Scaling gains can be included to scale r and/or e with 
no effect on the supporting theory, other than when defining projection domains 
as above. Likewise, the “scaling” can be generalized to stable dynamic filters for 
r and/or e with no effect on the supporting theory. In this way frequency shaped 
designs can be effected. 

The scheme described above can be specialized to the cases when Qf,Q are 
finite impulse response filters by setting n — 0. The , so defined, are stable for 
all bounded 0k . Also, either Qf or Q can be set to zero to simplify the processing, 
although possibly at the expense of performance. 

In the case that Qf is a moving average and Q is zero, then our scheme be- 
comes very simple, being a moving average filter Qf in series with the closed- 
loop system (AG, K). In this case then, if Qf is stable, guaranteed when the 
gains §k are bounded, and (AG, K ) is stable, then there is obvious stability of the 
adaptive scheme. 

When the linearized plant model AG* is stable, and one selects trivial values 
F, H = 0 so that K — 0, then the arrangement of Figure 2.6 simplifies to a 
familiar model-reference adaptive control arrangement depicted in Figure 2.9. 

In the case that Qf is set to zero there is no adaptive feedforward control action. 

The operators AG*, J k are in fact functions of the optimal trajectories x*. 
It makes sense then to have the operator Q also as a function of x*. Then the 
adaptive- Q approach generalizes naturally to a learning- Q approach as studied in 
a later section. 



Main Points of Section 

In the case of “smooth” nonlinear systems, linearizations yield trajectory de- 
pendent time- varying models. Straightforward generalizations of the adaptive- Q 
methods of Chapter 6 to the time-varying case allow application of the ideas to 
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enhance the performance and robustness of “optimal” nonlinear controllers. 



8.3 Stability Properties 

In this section we focus on stability results as a basis to achieve convergence 
results for our system. We first analyze a parameterization of the nonlinear plant 
AG with input A u and output Ay in terms of the coprime factorizations of the 
linearized version AG*, and stabilizing linear controller K* , and establish that 
this parameterization covers the class of well-posed closed-loop systems under 
study. Next, stability of the scheme is studied in terms of such parameterizations 
and then expected convergence properties are noted based on this characterization 
and known convergence theories in the linear time invariant case. 



Nonlinear System Fractional Maps* 

Let us consider the right and left coprime factorizations for the nominal linearized 
plant and controller, paralleling the approach used in Chapter 2 for linear systems, 
but here with time varying and at times nonlinear systems. The operators are ex- 
pressed as functions of the desired optimal trajectory x*, and optimal control u*, 
but since x*, u* are time dependent, then for any specific x*(-), «*(•) the oper- 
ators are merely linear time-varying operators, and can be treated as such. We 
denote AG as the (nonlinear) system with input A u and output Ay, and AG* is a 
linearization of the nominal plant G. Also, a unity gain feedback loop with open 
loop operator W a i is said to be well-posed when (I + W a i)~ l exists. Recall that 
for a nonlinear operator S, then, in general S(A + B) / 5/1 + SB, or equiva- 
lently superposition does not hold, and care must be taken in the composition of 
nonlinear operators. 

Theorem 3.1 (Right fractional map forms). Consider that the time-varying lin- 
ear feedback system pair (AG*, K*) is well posed and stabilizing with left and 
right coprime factorizations for AG*, K*, as in (2.1 1) and (2.12), and the double 
Bezout (2.13) holds. Then any nonlinear plant with AG such that (AG, K *) is a 
well-posed closed-loop system can be expressed in terms of a (nonlinear) operator 
S in right fractional map forms: 

A G(S) = N(S)M~ l (S) (3.1) 

-^AG + M 'S(J + M 'US) 'M '. (3.2) 

where 

N(S) = (N + VS), M(S ) = (M + US). (3.3) 



*The remainder of this section can be omitted on first reading, or at least the proof details which 
perhaps require technical skill, albeit paralleling developments for the linear case. 
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Also, closed-loop system (nonlinear) operators are given from 

T I -**rr I -K*V l [~t/ Mips 0l p U 1 
[-AG / J [-AG* / J + |_V Nj [o oj 1 N Mj ' 

(3.4) 

Moreover, the maps (3.1), (3.2) have the block diagram representations of Fig- 
ure 3.1(a) and (b) where 

Jg = \- M ] U M '1 ■ (3.5) 

|_ M~ l AG J 

The solutions of (3.1), (3.2) are unique , given from the right fractional maps in 
terms of AG, or (AG* — AG) as the (nonlinear) operator 

S = (-N + MAG)(V — U AG) -1 (3.6) 

= M(AG - A G*)M[I - U (AG - A G*)M]~\ (3.7) 



or in terms of the closed-loop system operators as 




Moreover, ( N(S ), M(S)) are coprime and obey a Bezout identity 

VM(S ) - UN(S) = /. (3.9) 

Proof. Care must be taken in our proof to permit only operations valid for non- 
linear operators when these are involved. Now simple manipulations allow (3.6) 
to be reorganized under the well-posedness assumption as 

[;]-[-» 

and via the Bezout identity, as 

\ M(S) ] | ^1 I = [ 'lil-K'AGT'V-'. 

[iV(5)J [N + VS] l N V] |_Sj L AG J 

(3.10) 

Thus under (3.6) then M~ X (S ) exists and, (3.1) holds as follows 

N(S)M~ l (S) = A G(I - K*AG)~ x V~ l [(/ - ^*AG) _1 y _1 ] 1 = AG. 
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To prove the equivalence of (3.1) and (3.2), simple manipulations give 
AG = AG* + (N + VS)(I + M~ l US)~ x M~ l - NM~ l 
= AG* + (V - AM _1 f/)S(7 + M~ l US)~ l M~ l 
= AG* + (V - M _1 lVf/)S(7 + 

= AG* + M~ l (MV - NU)S(I + M~ l 

= AG* + M~ 1 S(I + M ] US) 'M '. 

so that under (2.13), we have that (3.2) holds. Likewise (3.6) is equivalent to (3.7) 
as follows 



S = M(AG - AG*)(V - U AG) -1 
= M(AG - A G*)M(VM - UAGM)~ l 
= M(AG - AG*)M(I + UNM~ 1 M - U AGM)~ l 
= M(AG - AG*)M[7 - U(AG - A G*)M]~ l . 



To see that the operator of (3.1) is equivalent to that depicted in Figure 3.1(a), 
observe from Figure 3.1(a) that t = M~ x (e \ — USl) , or equivalently, l = 
(. M + US)~ l e. Also, (e 2 - w 2 ) = (N + VS)£ = (N + VS)(M + US)~ l e x , 
which is equivalent to (3.1). 

Now suppose there is some other (5 + AS) which also satisfies (3.1). Then 



M-[; 

= I’m g! T 7 1 (M + t/5 + t/A5) -i 

\_N yJ[5 + A5j 

Then, using (2.13), 

[-« 

= (M + G5 + GA5) -1 . 

[5 + AS\ 



(3.11) 



Premultiplication by [ / 0] gives M + US = M + US + U AS, and premultipli- 
cation by [ 0 / ] gives in turn that AS = 0. 

To verify (3.8), first observe that 

r 7 -t:*i _ r m -j/i r i oi \m+us oi 

[-AG 7 J _ \_-N V \ [-5 7j [ 0 yj 



(3.12) 
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FIGURE 
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Thus 

r -fit ' - K 'V 

|_-AG I J |_-AG* I J 



\m + us 


°1 


\> °T 


"m ol 


1" M 


-t'l 


[ « 


rj 


L-* 'J 


0 v\ 


-N 


vj 



M t/1 To ol [" M -if 
N v\ \_S 0 —N V 



and applying the double Bezout (2.13) gives 

r* -?]\\ — n ' -'fir- 

\_-N M J \_-AG I J |_“ AG * 1 J l~ N y J 

-e :]■ 

or equivalently (3.4) holds, and (3.8). (This result is generalized in Theorem 3.2.) 

Simple manipulations from Figure 3.1(b) give the operator representation of 
the G block to be J21 5(1 — J\ \ S’) -1 J\2 + J22, and substitution of (3.5) gives AG 
by (3.2). 

To establish coprimeness of N(S), M(S ) observe that under the double bezout 
(2.13) 

VM(S) - UN(S ) = VM -UN+ (VU - UV)S = I. 

Since I is unimodular, then from Paice and Moore (1990b, Lemma 2.1), 
N(S)M(S)~ l is a right coprime factorization. □ 

A number of remarks are in order. When AG is linear and time invariant, the 
above results specialize to the results in Chapter 2, although the details of the 
theorem proof appears quite different so as to avoid using superposition when 
nonlinear operators AG, S are involved. 

The fact that M,N,M,N,U,V,U,V are linear has allowed derivations to 
take place without differential boundedness or other such assumptions as in a 
full nonlinear theory as developed in Paice and Moore (1990a), Paice and Moore 
(1990b) using left coprime factorizations. 

Dual results apply for fractional mappings of K*(Q), as in (3.13) and (3.14) 
along with duals of the other results. Thus K*(Q) can be expressed as a linear 
controller K* augmented with a nonlinear Q. Also, by duality, Figure 3.1(a) de- 
picts a block diagram arrangement for 

K*(Q)=U(Q)V~ l (Qy, U(Q) = (U + MQ), V{Q)={V + NQ), 

(3.13) 
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where 

Q = (-£> + VK*{Q)){M - NK*{Q))~\ (3.14) 

Stabilization Results 

We define a system (G, K*) with G possibly nonlinear to be internally stable if 
for all bounded inputs, the outputs are bounded. 

Theorem 3.2. Consider the well-posed feedback system (AG, K*) under the 
conditions of Theorem 3.1, with AG and K* parameterized by Q, S as in (3.1), 
(3.13) and as depicted in Figure 3.1(a) and (b). Then the pair (A G*(S), K*(Q )) 
is well posed and internally stable if and only if the feedback system (Q, S) de- 
picted in Figure 3.2 is well posed and internally stable. Moreover, referring to 
Figure 3.1(c), the Jk, A G block with input/output operator T satisfies 

T = S. (3.15) 




FIGURE 3.2. The feedback system ( Q , S ) 



Proof. Observe that from (3.1 


L),(3.13) 








r / -K*(Q) 1 

|_-AG*(S) I \ 










r m ~u 

- \ _-N V 


]['. 


-el I ” M + US 

/JL o 


° r 

V + NQj 


(3.16) 


Clearly, under the double Bezout identity (2.13), 
(AG*, K*) well posed and internally stable. 


or equivalently 


under 


r / -k*(Q) i 

AG*(S) / J 


exists 


- [■ 


/ -el 1 

exists. 

-5 /J 


Equivalently, (A G*(S), K' 


"(0) is 


well posed if and only if (Q, S) 


is well 



posed. Thus under well-posedness assumptions, taking inverses in, and exploiting 
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(3.16), simple manipulations yield 

r / -^(G )] -1 

|_-AG*(S) / J 




Now internal stability of (AG*, K *) , (Q, S ), and stability of N, N, etcetera 
leads to internal stability of the right hand side and thus of (A G*(S), K*(Q)) 
as claimed. Moreover from (3.17), (2.13) 




Thus well-posedness and internal stability of (A G*(S), K*(Q)) and (AG*, K*) 
gives well-posedness and internal stability of (Q, S) to complete the first part of 
the proof. 

Now with J k defined as in (2.16), the operator T in Figure 3.1(c) can be rep- 
resented as 

T = V~ l AG(I - V~ X V AG)~ l V~ l - NV~ l 
= y _ 1 AG(y - U AG ) -1 - NV~ l 

= [y -1 AG — N + NV~ l UAG](V - GAG ) -1 ^ ^ 

= M[M~\V~ 1 + NV~ l U)AG - AG](y - GAG ) -1 
= M[AG - AG](y - GAG ) -1 
= S. 



A number of remarks are in order. This proof does not use superposition asso- 
ciated with operators Q, S, but does in regard to M, N, etc. The results following 
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Theorem 3.1 also apply for Theorem 3.2. Thus the proof approach differs (of ne- 
cessity) from the proof approach for the linear Q, S case based on work with the 
left factorizations , since when working with left factorizations, superposition is 
used associated with the operators <2,5. 

If || 5 || < e then by the small gain theorem for closed feedback loops, if || Q\\ < 
1/e then Q stabilizes the loop. From this, and Theorem 3.2 with (AG — AG*) 
suitably small in norm, there exists some Q which will guarantee stability. 

In the case where AG = AG*, we trivially have 5 = 0, and any Q se- 
lection based on identification of 5 will be trivially Q = 0. This contrasts the 
awkwardness of one alternative design approach which would seek to identify the 
closed-loop system as a basis for a controller augmentation design. 

Observations on examples in the linear AG case have shown that if A - * is robust 
for G, then 5 can be approximated by a low order system, thus making any Q 
selection more straightforward than might be otherwise expected. 

Averaging analysis could lead to robust convergence results as in the case of 
linear plants, but such is clearly beyond the scope of this work. 

Main Points of Section 

The closed-loop stability theory that is the foundation of the adaptive- Q schemes 
in the time-invariant linear model case is generalized in a natural way to the time- 
varying linear system context, and at least partially to the nonlinear setting. 

Example. Here we demonstrate the efficacy of the adaptive- Q approach through 
simulation studies. Consider an optimal control problem based on the Van der Pol 
equation. This equation is considered interesting because it is a simple yet funda- 
mentally nonlinear equation which has a rich oscillatory behavior, not in general 
sinusoidal. 



ii = (1 - xf)x\ - X2 + u\ X2 = xi, y = x i, (3.19) 
with xi(0) = 0, X2(0) = 1 and the performance index defined by 

1 r 5 

I = - / (x? + x? + u 2 )dt. (3.20) 

2 Jo 

A second-order algorithm of Imae and Hakomori (1987), using 400 integration 
steps, was adopted for the numerical solution of the open-loop optimal control 
signal u*. An arbitrary initial nominal control u = 0, t e [0, 5], was chosen. The 
value of the performance index was reduced to the optimal one in 4 iterations in 
updating n(-) over the range [0, 5]. 

Four situations have been studied in simulations. For each case we add a sto- 
chastic or a deterministic disturbance which disturbs the optimal input signal. 
Also, in some of the simulations we apply a plant with unmodeled dynamics. 
The objective is to regulate perturbations from the optimal by means of the in- 
dex A / = Jq (<5Xj + <$x| + 8u 2 )dt which is expressed in terms of perturbations 
8x,8u. For each of the disturbances added, and for the unmodeled dynamics case, 
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we compare five controller strategies, and demonstrate the robustness and perfor- 
mance properties of the adaptive- Q methodology. 

1. Open-loop design. 

Here we adopt the optimal control signal u* as an input signal of the non- 
linear system with added disturbance. Figure 3.3 shows that the open-loop 
design is quite sensitive to such disturbances in that x\, xj differ signifi- 
cantly from x*, x|. 




0 50 100 150 200 250 300 350 400 450 



Iterations 

FIGURE 3.3. Open Loop Trajectories 

2. LQG design. 

In order to construct feedback controllers, we adopt the standard LQG the- 
ory based on a discretized model obtained by fast sampling. This model 
is then linearized about the optimal trajectories and the performance index 
(3.20). Of course, the input signals u* + 8u are no longer ‘optimal’ for the 
nominal plant. The LQG controller’s design yields better performance than 
the open-loop case in that the errors x\ — x*, xi — are mildly smaller 
than in the previous figure for the open-loop cases, see Table 3.1. It is well 
known, however, that the LQG controller, although optimal for the nominal 
plant model under the assumed noise environment, may lose performance 
and perhaps its stability even for small variations from the nominal plant 
model. 

3. LQG/LTR design. 

In order to enhance the robustness properties of LQG controllers, we 
adopt well known loop transfer recovery (LTR) techniques Doyle and Stein 
(1979). Thus the system noise covariance Q in a state estimator design is 
parameterized by a scaler q > 0, and a loop recovery property is achieved 
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as q becomes large. In our scheme the state estimator ‘design system and 
measurement noise covariances’, Q*y(q) and RJ, are given by 

£}(<?) = / + <7 2 |y| [l o] ; R} = I, <7=50. 

There is a more dramatic reduction of errors x\ — x*, X2 — over that for 
the LQG design of the previous case as indicated in Table 3.1. Of course, 
the LQG/LTR design is identical to that for the case when q = 0. Also, 
simulations not reported here show that the LQG/LTR design performs vir- 
tually identically to an LQ design where states 8x are assumed available for 
feedback. 




0 50 100 150 200 250 300 350 400 450 

Iterations 

FIGURE 3.4. LQG/LTR/ Adaptive - Q Trajectories 



Disturbance 


d = 0.2 


d G U (0.1, 0.3) 


d G U( 0, 1) 


Open loop 


3.071 


3.056 


6.259 


LQG 


0.749 


0.744 


3.256 


LQG/LTR 


0.230 


0.228 


1.348 


LQG/Ad- Q 


0.348 


0.345 


1.993 


LQG/LTR/ Ad- Q 


0.160 


0.160 


1.001 



TABLE 3.1. A / for Trajectory 1, x(0) = [0 1 ] 
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The adaptive- Q, two-degree-of- freedom controller design for optimal con- 
trol problem is studied, with the LQG or LQG/LTR controller K and the 
adaptive = [ 2/ Q ] using least square techniques. Third-order FIR mod- 
els are chosen for the forward Q f(z) and the backward Q(z). Simulations, 
summarized in Table 3.1, show that adaptive - Q controller design strength- 
ens the robustness/performance properties of both the LQG and LQG/LTR 
design without the need for any high gains in the controller. See also Fig- 
ures 3.3 and 3.4. The intention in this first design example has not been 
to demonstrate that an adaptive- Q approach works dramatically better than 
all others, although one example is shown where such is the case. Rather, 
we have sought to stress that the adaptive- Q method is perhaps best used 
only after a careful robust fixed controller design, and then only to achieve 
fine tuning. Actually, for the design study here, the robust LQG/LTR de- 
sign performed better than the LQG adaptive- Q design. The values of A / 
for all five cases are summarized in Table 3.1 for a deterministic distur- 
bance d = 0.2, and then two stochastic disturbances, in the first instance 
with d uniformly distributed between 0.1 and 0.3, and in the second with d 
uniformly distributed between 0 and 1 . 

To demonstrate the robustness of the adaptive- Q control strategy, the sim- 
ulations were repeated with unmodeled dynamics in the actual plant, see 
Table 3.2. The state equations of the actual plant in this case are 

X\ = (1 - xf)xi -X2 + X3 + U, 

X% = XI, 

X I = -X3 - 4X3 + « , 

y = xi, 

with initial state vector [ o t 0 ]. 

The simulations in Tables 3.1 and 3.2 are repeated for different initial con- 
ditions, and thus a different optimal trajectory. The results are included in 
Tables 3.3 and 3.4. This trajectory also has the same unmodeled dynamics 
added to demonstrate robustness. 



Disturbance 


d = 0.2 


Open loop 


5.9077 


LQG 


1.943 8 


LQG/LTR 


0.662 3 


LQG/ Ad- Q 


0.984 


LQG/LTR/ Ad- 2 


0.425 1 



TABLE 3.2. A / for Trajectory 1 with unmodeled dynamics, x(0) = [ 0 1 0 ] 
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Disturbance 


d = 0.2 


d e U (0.1, 0.3) 


d e U( 0, 1) 


Open loop 


3.664 6 


3.647 8 


7.4502 


LQG 


0.7524 


0.7476 


3.3817 


LQG/LTR 


0.2165 


0.2148 


1.2766 


LQG/Ad- Q 


0.3447 


0.3415 


1.9734 


LQG/LTR/ Ad- g 


0.1505 


0.1495 


0.927 8 


TABLE 3.3. A / for Trajectory 2, jc(0) = [ 


10.5] 



Disturbance 


d = 0.2 


Open loop 


4.730 1 


LQG 


1.2805 


LQG/LTR 


0.5162 


LQG/Ad- Q 


0.6313 


LQG/LTR/ Ad- 2 


0.298 1 



TABLE 3.4. A I for Trajectory 2 with unmodeled dynamics, x(0) = [ 1 0.5 0 ] 

In our simulation for the adaptive- Q controller, two passes are needed for 
“warming up” of the controller. Subsequently, the coefficients in Qf and 
Q , in the notation of (2.21), “converge” to slowly varying values in the 
vicinity of y = 0.097 6, yi = —0.0002, y 2 = —0.1016, fi = —11.18, 
Pi = -9.247, p 2 = -7.891, with a t = 0. 

The prefilters P\ 2 M used in our study are as follows: 

+ BF ^ + Bu\ It = Q ^ + Q «*, 

with input u* and output . Likewise for the prefilters driven by Sr and s. 
Our simulations not reported here show significant improvements when 
scaling adjustments are made to r and e. Also, other simulations not re- 
ported here show that there is insignificant benefit with increasing the di- 
mensions p = 3, m — 3, n = 0 in Q, although the cost of reducing p or 
m is significant. 

The message from this simple example is clear. Open-loop optimal control of 
nonlinear systems is nonrobust. Linearization and application of linear optimal 
feedback methods improves the situation. Working with robust linear methods is 
even better, and adding an adaptive- Q loop enhances performance and robustness 
even further. 
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8.4 Learning- Q Schemes 

For robust nonlinear optimal control, it makes sense to explore the adaptive- Q 
enhancement approach modified such that Q is state (or state estimate) dependent. 
Such an approach is termed here learning- Q control. 

In generalizing the least squares based adaptive- Q filter coefficients update 
scheme, applied for the nonlinear control tasks of the previous sections, the es- 
sence of our task is to replace a parameter least squares algorithm by one in- 
volving functional learning. To optimize the 0-filter when its parameters are 
state dependent, the key strategy we propose here is to apply functional learn- 
ing algorithms as in Perkins, Mareels and Moore (1992). The least squares 
based functional learning of Perkins et al. (1992) suggests bisigmoid sum rep- 
resentations of the parameters over the state space, or for practical reasons, 
over only the significant components of the state space. The parameters of this 
representation are tuned on line by a least squares scheme. Bisigmoids, such 
as Gaussians or truncated Gaussians, B-splines, or radial basis functions have 
the dual roles of interpolating between parameter estimates at grid points in 
the state space, and of spreading learning on either side of a trajectory in the 
state space. 

In adaptive control, the controller for the plant adapts to optimize some per- 
formance specification. Should the plant be time-varying, then the adaptive con- 
troller tracks in some sense an optimal controller. There is built into the con- 
troller a forgetting factor so that distant past experiences are totally forgotten. 
If the plant dynamics are nonlinear being a function of a slowly changing vari- 
able , such as the slow states of the plant, then it makes sense to remember the 
past in such a way that the controller can recall appropriately from past experi- 
ence and give better performance than it would with built-in forgetting. To facil- 
itate such control action, enhanced with memory, functional learning algorithms 
appear attractive. 

Functional learning here refers to a method by which the values of a function 
y = f(x) can be estimated at all points in the input variable space T x from data 
pairs ( Xi , y,), or noisy measurements of these. Given an estimate /(•) at time k, 
denoted /, then with a new measurement Xk, a prediction of yk is % = f(xk). 
The error ( y—% ) can then be used to update /(•) for time k+ 1 based on assumed 
smoothness properties of /(•)• The function is here represented as a sum of simply 
parameterized functions which could be basis functions such as polynomials or 
Gaussians. We define the error between a representation of a given function /(•) 
and the actual function /(•) in terms of some error norm. Thus, here, the learning 
involves adapting the parameters of the basis functions to achieve lower error 
norms. We look to approximate arbitrary continuous functions within a class of 
such. 

A key representation theorem for our approach is in Cybenko (1989). This the- 
orem tells us that sums of sigmoids, or more general bisigmoids such as Gaussians 
or truncated Gaussians, suitably parameterized, are dense and can represent func- 
tionals over finite domains with arbitrary precision. 
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Function Representation 

Given a function, /(•), an approximation to that function could be represented 
by a superposition of a finite number of the simply parameterized functions /;(•), 
such as sigmoids or bisigmoids, each centered at different points , within the in- 
put variable T* space. The representation must be chosen with regard to required 
accuracy, convergence properties and computability. For example, to approximate 
a two input variable scalar function with a bounded first derivative by a grid of 
simply parameterized functions being piecewise constant functions on a grid, the 
following result is easily established. 

Lemma 4.1. Suppose there is given a two input variable scalar function f(x,y ) 
with a first derivative bounded by C, and a square region R = {(x, y) : \x \ , |y| < 
r] over which it is to be approximated. Furthermore, suppose there is an approx- 
imation to the function by a piecewise constant function on a rectangular N x N 
grid covering the region R. Then the I2 error bound between f(x, y) and the 
approximation f(x,y) is 



Proof. 



( 4 . 1 ) 



_ ryo+r/N rxo+r/N 

b = J2 / (/(*. y) - /(*. y» dx d y • ( 4 - 2 ) 

N i J yo 4 X0 

The worst case approximation to f(x, y) of a level function will be when f(x,y) 
is at its maximum gradient within a grid square. Also, the worst error will be 
the same for each grid square. To calculate a worst case error then, consider the 
“worst case” functions f(x, y) = fo ± Cx ± Cy. In this case, 

ryo+r/N rxo+r/N 

b = N 2 / (f(x, y) - f(x 0, y 0 ) ± Cx ± Cyfdx dy. ( 4 . 3 ) 

J yo Jx 0 

Now, the region of that square is [x,y : x 0 < x < xo+r/N, yo < y < yo+r/N}. 
Set / = f(x 0, yo)- Then a substitution of variables gives 

b= 1 - N 2 C 2 (^-) 4 =0 ( ^). ( 4 . 4 ) 

6 N N-t- 

□ 

In selecting the simply parameterized functions f(-) for learning in the control 
environment of interest, we take into account the need for ‘fast’ learning, reason- 
able interpolation and extrapolation approximation properties, and the ability to 
spread learning in the T x space. For ‘fast’ learning, we require here that the mea- 
surements are linear in the parameters of /,■ (•) so that the least squares techniques 
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can apply. This contrasts the case of neural networks where backward propagation 
gradient algorithms are inevitably ‘slow’ in convergence. 

For reasonable interpolation capabilities, any of a number of selections such 
as polynomials, splines, sigmoids, or bisigmoids can be used, but to avoid poor 
extrapolation outside the domains of most excitation of Xk in T x , we select bisig- 
moids. Likewise, only the bisigmoids allow learning along the trajectories in T x 
to be spread acceptably to neighborhoods of such trajectories. This approach is 
taken in Perkins et al. (1992) for an open loop identification task, which we now 
adapt for our control task. First let us review the least squares learning taken in 
Perkins et al. (1992), and related results. 

Least Squares Learning 

Consider the signal model usually derived from an ARMAX representation, 

y k = <t>' k @(x k ) + w k , (4.5) 

where yk are the measurements, is a known regression vector of the model 
inputs and outputs, and ©(•) are the unknown functionals with input variables Xk, 
representing the perhaps nonlinear ’parameters’ of an ARMAX model. Here co k 
is taken to the zero mean white noise. 

Let us investigate finite representations estimating ©(x) of the form 



©(x) = J2 K i( x ’ ViMVi) = K'j(x)@( r 7 ), (4.6) 



with @'(r 7 ) = [O’ in) . . . 0'(y„)], and K’j{x) = [K'j(x, yi) . . . Yn)l Here 
0(Yi) are the parameters and X/(x, y,) the interpolation function representation 
(4.6). For simplicity we work with Kj(x, y,-) which acts as both a scalar inter- 
polating function and learning spread function between the points x & T x and 
x e T/ with a preselected set of points T / = {yi, Y 2, . ■ . , Yn } in T*. In our simu- 
lations, we use Gaussians or truncated Gaussians for the vectors indicated above. 
Now (4.5) can be represented as 

y k = <t>(x k y®(T I ) + a> k , (4.7) 



where 



<F(x*) = KtixkY t> k , ®{x k ) = Kjixk)®^!), (4.8) 

and u>k approximates Wk- 

Consider an error measure for the representation: 

4 r) m = || 0 (**) - ‘ • 



(4.9) 
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As shown in Perkins et al. (1992), under strong persistence of excitation condi- 
tions on Xk minimization of this index is equivalent to minimization of the <© 
index : 



d 2(®) = [/£« - W(.v) | 2 r/.vj ' . (4.10) 

A key result associated with this latter minimization task is as follows: 

Theorem 4.2. The minimization task has a unique critical point, denoted &*if 
and only if the elements of K/(x) are allowable, in that 

oo > KjMK'jWdx J>0. (4.11) 

This optimal © is given from 

©* = K I (x)K' I (x)dx^ Jr y(x)K'(x)dx. (4.12) 

Moreover, when Q(x) is reconstructive with respect to the class of functions ©(x) 
of (4.6), then 0(x) is uniquely parameterized as in (4.6) with © = ©* given in 
(4.12). 

Proof. The proof of this can be found in Perkins et al. (1992). □ 

In order to minimize d^ of (4.9) for r = 1,2,..., given a sequence {x*, yk }, 
standard least squares derivations applied to (4.9) and (4.10) lead to a recursive 
estimate of ©(X©, denoted ©*(x©, as 

©*(x© = K’jixVWi) (4.13) 

©ft(r© = ©*- 1 ( 1 ©) + Pk(T I )<b,(xk)[yk - $ , / (x*)©*_i(r / )], (4.14) 

where 

P^ 1 {T I ) = P t f\{T !) + <$> i {xkWjixk), <&(**) = */(*©**, (4.15) 

with suitable initial conditions ©, P. Under appropriate conditions (Perkins et al. 
(1992)), Pf 1 approaches a diagonal matrix. With truncated Kj we have that Pf l 
is block diagonal with only one block updated at each iteration and only one 
corresponding segment of ©* updated. In selecting a truncation, there is clearly a 
trade off between function estimation accuracy and computational effort. 

Least Squares (Scalar variable case) 

Now with the definitions of (2.20), and as shown in Tay and Moore (1991) for the 
case of scalar variables 8uk, £>>’*, r*, bk (for simplicity), with s* = Q fu* k + Qrk, 




8.4. Learning- Q Schemes 235 



following the derivations leading to (2.24) we see that is linear in 0, as 
& = «>*© + «*, (4.16) 

= [(e*-l/*-l - 6fc-l) ••• (h-n/k-n - Zk-n) ~Hk ■■■ -§fc-m] • 

(4.17) 

These equations allow a least squares recursive update for 0, denoted ©£, to 
minimize the index E* =1 || e,|(-) || 2 as spelled out earlier for the adaptive- (2 scheme. 
Here et\® denotes e j ^ i ^ i@ in obvious notation. In fact the details are a special 
case of those now derived for the proposed learning- Q scheme of Figure 4. 1 . Note 
that the adaptive- Q scheme is that of Figure 4.1 specialized to the case when 
©jt(x*) is independent of x*, so that ^(x|) is replaced by &• 




FIGURE 4.1. Two degree-of-freedom learning- Q scheme 



Learning- Q Scheme based on x* 

We build upon the earlier work of Chapter 6 and this chapter by extending the 
adaptive- Q scheme to what could be viewed as an adaptive- Q (x*) scheme, but 
which we call a learning- Q scheme. The filter with which we work has co- 
efficients which are functions of the state space. Denoting k(x). oiikM, Pik(x), 
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the key idea of the learning - Q algorithm is to update estimates a(-), for all x 
in some domain T x in which lies. The filter is implemented as a(x|). The 
arrangement is depicted in Figure 4. 1 . The least squares functional learning block 
yields estimates 0 (xt ) for implementation of the filter k(x* k ), being driven from 
f*, ek and x%. The parameter estimates &k(x k ) are derived via the approach 
above. Thus , corresponding to (4.7) , we have the formulation (4.16) , and corre- 
sponding to (4.13)-(4.15) we have 

©*(*£) = £/(x*;)®*(r/), (4.18) 

©t(r /) = % i (r /) + p k ( r/)4» 7 (*p [&(**) - 4>;©(r 7 )] , (4.19) 

Pk 1 = P k-r(Fr) + <!>/(**)«>'/(**). (4.20) 

The filter can be implemented as a(xa), where x k = x k + 8xk, as an al- 
ternative to implementing k(x* k ). This suggests also that the nominal plant AG, 
controller K and indeed J be functions of x t , rather than x*. For this case then, 
any A G(x t ), K(x t ), J(x t ) are inevitably nonlinear and a nonlinear factorization 
theory is required. Care should be taken because there is in essence an extra feed- 
back loop in the system which can cause instability, so this approach is not rec- 
ommended. 

Of course when 8x is small, one expects that x* could be just as good an es- 
timate of x t as Xf In this case, there would be an advantage in working in a full 
nonlinear context. However, we would expect 8x to be small only when the plant 
is nearly linear, and to avoid dealing with a full nonlinear context is really to avoid 
tackling systems that are in essence nonlinear. 

Learning- Q Simulation Results 
The Signal Model and Performance Index 

Consider again the specific nominal plant model (Van der Pol equation): 

Go : x\ = (1 — xf)xi - X2 + u + d; X2 = x\, y = x\, (4.21) 

with scalar input u, scalar output y, and state vector x = [*i *2 ]'. Consider also a 
regulator performance index defined by 

1 

I(x 0, u) = - / (*? + *! + u 2 )dt. (4.22) 

2 Jo 

Of course for such a simple example, one could use a trivial linearization, by tak- 
ing u = x\x\+u\ and then optimizing u\ — Kx viaLQG control, thereby achiev- 
ing an attractive nonlinear controller. Also, for more general nonlinear systems 
one could similarly exploit the linearization approach of Isidori (1989). However, 
here we wish to illustrate the design approach of the previous sections and use 
this plant as an example. Here, for each initial condition investigated, an opti- 
mal trajectory is calculated. Next, the closed-loop feedback controller schemes of 
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the previous section are studied in the presence of constant and stochastic distur- 
bances added to the plant input, and some of the simulations include unmodeled 
dynamics. The actual plant with unmodeled dynamics is 



X\ = (1 - X2)X\ - X2 + X3 + u + d, 
X2 = Xl, 

X 3 = — X3 — 4X3 + U, 



(4.23) 



where X3 is the state of the unmodeled dynamics and d is the disturbance. 

The major objective of the learning - Q method is to learn from one trajectory, 
or set of trajectories information which will enhance the control performance for 
a new trajectory. 

For the simulations, functions 0(x) = 0(xi , X2) are represented as the sum of 
equal covariance Gaussians centered on a sparse two-dimensional grid in r xi?X2 
space. In the learning- Q scheme, the weighting of the Gaussians is updated to 
best fit the given data via least squares functional learning. The tailing off of the 
Gaussians will ensure that trajectory information effectively spreads only to the 
neighboring grid points with more weighting on the near neighbors. 



Selection of Algorithm Parameters 

The algorithm requires a priori selections of the interpolating functions and their 
parameters including location on grid points in state space. It then learns the rel- 
ative weightings. 



Selection of Grid Points 

The state space of interest is selected to include the space spanned by the optimal 
trajectories. The Gaussians are fixed initially at a 4 x 4 grid of yi over a unit 
“box” region covering well the trajectory region to avoid distortions due to edge 
effects. A scaling method facilitates quick changes of the apparent denseness and 
compactness of the grid. Optimal placing of the grid points for one particular 
trajectory is not usually optimal for other trajectories, and the chosen grid points 
can be seen in Figure 4.2. 

The Spread of the Interpolating Function 

The interpolating function is of the form We~ n d ‘ k , where n is the number of 
Gaussians in each dimension, dj is ||x — y,- 1| 2 , W is the initial weighting, and k is 
a constant used to tune the learning (in our simulations W — 10 -10 , k = 4). The 
“optimal” spread of the Gaussian is a function of the shape of the function being 
learned, and the denseness of the Gaussians in the grid. 



Trajectory Selection 

The data must be “persistently” spanning the grid space in order to leam all the 
Gaussian weights. As the estimates are functions of stochastic outputs, greater 
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1.5, 



0.5 

0 



-0.5 

_1 1 

-1 -0.5 0 0.5 1 1.5 

FIGURE 4.2. Five optimal regulation trajectories in r xl X2 space 

excitation of a mode allows for a more accurate estimate of the weighting of that 
mode. Five initial conditions have been chosen to illustrate performance enhance- 
ment due to the learning- <2 approach. The initial conditions of x are: (0.5, 1), 
(0.5, 0.5), (1, 0.5), (0, 0.5), (0, 1). The optimal regulation state trajectories cal- 
culated from these initial conditions are shown in Figure 4.2, to indicate the extent 
to which the state space T x is covered in the learning process. 

Results 

For the trajectories and algorithm parameters of the previous subsection, robust- 
ness and performance properties are examined. 

Persistence of Excitation 

In order to test the learning, two simulations were compared. In the first, de- 
noted Global Learning , Trajectories 1 through 5 were executed and then re- 
peated, each time enhancing the learning with the knowledge previously learned. 
In the second, denoted Local Learning, Trajectory 5 was repeated 10 times, to 
achieve enhanced learning. In the example of Table 4.1 below, the disturbance 
was d = 0.2. 




Run Number 


Global Learning Local Learning 


5 


0.337 3 


0.3422 


10 


0.3296 


0.335 5 



TABFE 4.1. Error index for global and local learning 



As can be seen in Table 4. 1, the global learning actually gives marginally better 
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results than the specialized local learning by virtue of its satisfying persistence 
of excitation requirements. These are typical of other simulations not reported 
here. 



Deterministic Disturbances 

There are three types of disturbances simulated. First a zero disturbance is used, 
and since the plant then follows the optimal trajectory, as expected the values of 8x 
and 8u are zero. The other disturbances used are: a constant co = 0.2, stochastic 
disturbances uniformly distributed, and disturbances where the disturbance is a 
function of position in state space. 

Since the error index is a function of the total level of input disturbances, the 
constant disturbance is the one used to compare various parameters and methods, 
with the stochastic and functional disturbances being then used to test the selected 
parameters under more realistic conditions. 



Stochastic Disturbance 

The simulation is run for each trajectory with d = RAND(— 0.5, 0.5), that is 
the disturbance is uniformly distributed with an upper bound of 0.5, and a lower 
bound of —0.5. In the case of ‘global learning’, where Trajectories 1 through 5 
were run, then repeated for all of the trajectories, the algorithm gives an improve- 
ment in the error index, except Trajectory 5. Details are summarized in Table 4.2. 



Trajectory 


Run 1 


Run 2 


Improvement 


1 


0.015 7 


0.008 6 


45% 


2 


0.0160 


0.007 0 


56% 


3 


0.063 7 


0.021 3 


66% 


4 


0.070 8 


0.012 3 


83% 


5 


0.0113 


0.019 9 


-5% 



TABLE 4.2. Improvement after learning 



Unmodeled Dynamics 

The simulations were run with the disturbances as before, as well as with the 
inclusion of the unmodeled dynamics as in (4.23). The system achieved good 
control, with for example the error indices of Trajectories 1 through 5, then re- 
peated in a stochastic disturbance case being 2.8, 0.9, 1.5, 1.1, 1.6 and the second 
run giving 1.7, 1.0, 1.5, 1.1, 1.7. 
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Nearest Neighbor Approximation and Grid Size 

The standard algorithm requires O (n 2 ) iterations for calculation of the values of 
an n x n grid. An approximation can be made where only the weights of the clos- 
est Gaussians are updated at each step. This approximation significantly speeds 
the algorithm, but loses accuracy. As is shown in Table 4.3, for the case with a 
stochastic disturbance d = RAND(— 0.5, 0.5) as above, and with no unmodeled 
dynamics, however going to a finer grid of 5 x 5 Gaussians, with the nearest neigh- 
bor approximation/truncation improves upon the 4 x4 full calculation/untruncated 
case. 



Average 


4x4 truncated 4x4 5x5 truncated 


Run 1 
Run 2 


0.3970 0.3844 0.3828 

0.345 8 0.3451 0.3304 



TABLE 4.3. Comparison of grid sizes and approximations 



As expected, the finer grid improves in comparison to the others during the 
second run, as it is better able to fit to the information given. The © / surfaces 
generated in this simulation for the 2 x 2, 3 x 3, 4 x 4 and 5x5 cases for the 
constant disturbance d — 0.2 are shown in Figure 4.3. The error index averaged 
over the second run for these cases are respectively, 0.3204, 0.325 2, 0.3465, 
0.323 0. The 4x4 case gave the worst result, possibly due to the artifact on the 
surface not displayed by the others. These results show that a finer grid spacing 
may not always improve the control. 



Comparison with Adaptive case 

Let us compare the results of our learning controller with those of the adaptive- Q 
controller . In the first instance, when running a trajectory for the first time, the 
adaptive algorithm gives better results than the learning algorithm. Allowing the 
learning algorithm previous experience on other trajectories however, lets it in 
most cases “beat” the adaptive one. For instance, in the nonzero mean stochastic 
disturbance case {<y e 0.2 ±0.1}, with no unmodeled dynamics, for Trajectory 1, 
the adaptive case gave 0.490 7, with the learning case giving 0.367 1 after learning 
on some trajectories. 

However, the adaptive case can also be enhanced to give better performance 
than the learning scheme by exploiting previous information from a learning- Q 
scheme. Simulations are performed comparing the extended adaptive to the learn- 
ing scheme with a variety of disturbances and with/without unmodeled dynamics. 
The constant, stochastic, and nonconstant deterministic disturbances are d — 0.2, 
de 0.2 ± 0.05, d = x\ ± x 2 . The results are summarized in Tables 4.4 and 4.5. 
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Main Points of Section 

Learning- Q schemes are really adaptive- Q schemes with Q being a parameter- 
ized nonlinear function of the state. Adapting its parameters on line achieves 
functional learning. In our example, the benefit of a learning- Q scheme over an 
adaptive- Q scheme would be worth the computational cost only in performance 
critical applications. 



8.5 Notes and References 

The application of the direct adaptive- Q techniques to nonlinear systems was first 
studied in Imae et al. (1992) The generalization to learning- Q schemes was first 
studied in Irlicht and Moore (1991). At the heart of this work is some form of 
functional learning. The application of parameter update techniques to functional 
learning theory is of course the meat of neural network theory. In these, the pa- 
rameterizations are fundamentally nonlinear. Here in contrast and for simplicity 
our focus has been on nonlinear functions which are linear in their parameters, 
such as in a Gaussian sum representation, as explored by us in Perkins et al. 
(1992). Thus here our learning-g filter, based on Irlicht and Moore (1991), uses 
such representations. 

This work on adaptive- Q and learning- Q techniques provided motivation for 
us to understand further the theory of nonlinear fractional maps and nonlinear 
versions of the Youla-Kucera parameterizations, see Paice, Moore and Horowitz 
(1992), Perkin s et al. (1992) and Moore and Irlicht (1992). 

One key insight to cope with nonlinear systems is that in the nonlinear case 
where the operators are initial-condition dependent, the mismatch between those 
of estimators/controllers and those of the plant can be viewed as unmodeled dy- 
namics, namely as a nonlinear operator S. 

Another observation which helps explain much of the scope of the nonlinear 
theory is that the class of all stabilizing controllers for nonlinear plants is more 
easily characterized in terms of left coprime factorizations, yet these factorizations 
are much harder to obtain than right coprime factorizations. 




CHAPTER 9 



Real-time Implementation 



9.1 Introduction 

We have until this point, developed the theory and algorithms to achieve high 
performance control systems. How then do we put the theory into practice? In 
this chapter, we will examine real-time implementation and issues in the in- 
dustrial context. In particular we will discuss using discrete-time methods in a 
continuous-time setting, the hardware requirement for various applications, the 
software development environment and issues such as sensor saturation and finite 
word length effects. Of course, just as control algorithms and theory develop, 
so does implementations technology. Here we focus on the principles behind 
current trends, and illustrate with examples which are contemporary at the time 
of writing. 

We begin with the hardware requirements. The hardware for a modern con- 
trol system is microprocessor-based. Analog-based control systems are still 
used in some speed critical applications, and where reliability is already estab- 
lished and is critical. However the cost in implementing complex algorithms 
using analog circuits with present day technology is not competitive with the 
new digital technology available. With modem fast microprocessors and digi- 
tal signal processors (DSPs), there is the possibility of connecting many pro- 
cessors to work in a parallel fashion, and it is now possible to implement 
fairly complex algorithms for sampling intervals of microseconds. The speed 
consideration is therefore becoming less important and digital implementation 
dominates. 

We begin the chapter with a discussion on using discrete-time methods for 
continuous-time plants. The key theorem is the Nyquist Sampling Theorem. 

Our next task is the design of a stand-alone microprocessor-based control sys- 
tem. Design and implementation of such a system can be time consuming in prac- 
tice. This is because there is very little scope for incremental testing and verifica- 
tion in the initial stage of the design. The engineer has to get both the hardware 
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and software working at the same time in order to verify that the design is success- 
ful. Often when the system fails to work, the engineer is left wondering whether 
the bug is in the hardware or the software. In response to this difficulty, many 
microprocessor manufacturers are now marketing emulator sets that essentially 
provide a software development and debugging environment. However such sets 
tend to be very expensive and are available for popular microprocessor series only. 
Also, there may be advantages in choosing certain specific processors to exploit 
their on-chip facilities for a particular application. In such cases, we may be left 
developing the controller system without the aid of emulator sets. We will outline 
in this chapter a systematic procedure that will greatly simplify the task of devel- 
oping a stand-alone system without the aid of an emulator set. Design examples 
are also provided. 

To cope with the rather complex algorithms introduced in this book, there 
is a need to look into high performance hardware platforms. We will describe 
in this chapter a two-processor design involving a number crunching oriented 
DSP and an input/output oriented microcontroller. We will highlight the mod- 
ular approach adopted to facilitate easy development of both the hardware and 
the software. 

It is well understood that the cheapest systems are ones which use the least 
number of components and at the same time use components that are commonly 
available. Take for example, a personal computer mother board which implements 
a feature-rich complex computing environment. This can be purchased for a rel- 
atively low price because the personal computer mother boards are produced by 
the millions and great effort is made to reduce component cost. Also with highly 
automated assembly lines, possible only with high volume production, the per 
unit manufacturing cost of such a board is very low. A comparable customized 
system with production volume of only a few thousands would be many times 
more costly. 

The plunging cost of mass-produced electronic boards gives rise to some new 
thoughts on designing the hardware of a control system. The trend is the design 
and manufacture of modules that are feature-rich so that they can be mixed and 
matched into systems for use in many applications. This allows the production 
volume of each module to be high and therefore drives the per unit cost down. 
An immediate implication is therefore to design a universal input/output interface 
card that has many features to cater to all kinds of controller needs. The specific 
control system is then assembled from ready-made personal computer mother 
boards and the input/output interface card. We will describe the design of one 
such card in this chapter. 

Development of the software for control algorithms is a parallel area for at- 
tention alongside hardware development. For simple controllers, this is not an 
issue. However, the high performance controllers studied in this text are complex 
enough for software aspects to be important. In this chapter we will look at the 
various platforms for which such software can be developed easily. The intention 
is to aid the designer to take steps towards realization, or at least to be able to talk 
to the implementation specialist. 
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9.2 Algorithms for Continuous-time Plant 

For the majority of controller designs introduced in this book, plant mod- 
els which are discrete-time, linear, time-invariant models are used, and con- 
sequently, the resulting controllers are discrete-time, linear, and time-invariant 
controllers. The discrete-time setting is desirable since we believe that in vir- 
tually all future controller realizations, digital computers and discrete-time cal- 
culations will be used. The discrete-time model of (2.2.3) can be directly de- 
rived from the underlying process in some plants, or indirectly by an identifi- 
cation scheme. Of course, for most plants, the underlying process is analog in 
nature, and indeed corresponding results to those spelled out in this book can 
be derived for continuous-time models, with little difficulty. How then do we 
deal with continuous-time plants when our approach is via discrete-time theory 
and algorithms? 

For continuous-time plants and digital computer controller implementation, 
there are two approaches. In the first approach, a continuous-time controller de- 
sign is based on the continuous-time plant model. The continuous-time controller 
is then replaced by an approximated discrete-time controller for implementation 
on a digital computer with appropriate analog-to-digital, and digital-to-analog 
converters. However in general, a relatively fast sampling rate has to be used in 
practice in order to achieve a good discrete time approximation. For recent work 
in this direction, see Blackmore (1995). 

An alternative approach is to first derive a discrete-time plant model from 
the continuous-time plant with its analog-to-digital and digital-to-analog con- 
verters, anti-aliasing filters, and post sampling filters attached, and then per- 
form a discrete-time controller design based on the discrete-time model of 
the plant, see Figure 2.1. This approach can lead to a possible reduction in 
the sampling rate compared to the former approach, see Astrom and Witten- 
mark (1984). We will adopt this approach in the treatment of continuous-time 
plants. 




FIGURE 2.1. Implementation of a discrete-time controller for a continuous-time plant 
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Let’s consider the following linear, time-invariant continuous-time plant model. 
x(t) = A c x{t) + B c u{t), 
y(t) = C c x(t) + D c ii(t). 

The pairs (A c , B c ) and (C c , A c ) are assumed stabilizable and detectable, respec- 
tively. 

This continuous-time plant model (2.1) can be discretized in time by an appro- 
priate sampling process. For the moment, assume that the same sampling inter- 
val is used for both the inputs and outputs of the plant and zero order holds are 
inserted at each input to the plant to achieve digital-to-analog conversion. This 
arrangement is depicted in Figure 2.1. 

For the continuous-time plant (2.1) embedded in the scheme of Figure 2.1, with 
sampling time T, the discrete-time model (2.2.8) then has state space matrices as 
follows. 

A = e AcT , B = [ e AcS dsB c , C = C c , D= D c . (2.2) 

Jo 

With this discrete-time model, discrete-time controller design techniques can be 
applied and the control scheme implemented as in Figure 2.1. In this arrange- 
ment, the control signal derived from the controller accesses the plant through the 
digital-to-analog converters. 

It is also possible to design discrete-time controllers directly for continuous- 
time plants. These methods are based on a lifting idea that exploits the periodic 
nature of the sampled data controller when applied to a continuous-time sys- 
tem, Feuer and Goodwin (1996). Alternatively one can work with fast sampling 
model representations of the plant while performing the design at lower sampling 
periods, Keller and Anderson (1992), Madievski, Anderson and Gevers (1993), 
Blackmore (1995), Gevers and Li (1993), Williamson (1991). We do not explore 
these approaches here. 



9.3 Hardware Platform 

In this section we explore a microcontroller-based solution, a dual processor so- 
lution for fast processes, and a personal computer based solution for achieving 
controller hardware platforms. 

A Microcontroller-based Solution 

The block diagram of a typical real-time controller system is shown in Figure 3.1. 
The system can be divided into three subsystems. They are the computing section, 
the input/output interface subsystem and the host computer interface subsystem. 

The main subsystem of the controller is for computing. It consists of the target 
microprocessor, nonvolatile memory such as Erasable Programmable Read-Only 
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FIGURE 3.1. The internals of a stand-alone controller system 



Memory (EPROM), volatile memory such as Random Access Memory (RAM) and 
a timer circuit. This subsystem is responsible for executing the software codes of 
the control algorithm as well as controlling all the other devices connected to the 
system. The software codes that implement the control algorithm are stored in the 
EPROM. The size of the EPROM will depend on the complexity of the algorithm 
implemented. The RAM is required to store all temporary variables in the exe- 
cution of the control algorithms. For the implementation of a simple controller 
scheme such as the ubiquitous proportional plus integral plus differential (PID) 
controller, a RAM size of about 20 bytes is sufficient. For more complex algo- 
rithms, the size increases correspondingly. The timer circuit is included to allow 
implementation of the sampling time interval. 

The input/output subsystem consists of interface circuits to allow the micropro- 
cessor to communicate with sensors and actuators of the plant. The exact circuit 
will depend on the sensors and actuators used. In the case of industrial processes, 
where the sensor instruments usually deliver sensor readings in the form of ana- 
log signals in the range of 4 mA to 20 mA, the input interface circuits then con- 
sist of analog-to-digital converters and current-to-voltage converters. In the case 
of robotic applications, where the sensor instruments are optical encoders that 
deliver quadrature phase pulses, the input interface circuits consist of up/down 
counters. 

As for actuators, industrial processes usually have actuators that accept ana- 
log signals in the range of 4 mA to 20 mA. The output interface circuit are then 
digital-to-analog converters together with voltage-to-current converters. On the 
other hand, in servo-mechanism type processes where DC motors are used, the 
interface circuits are required to generate pulse-width-modulated (PWM) signals. 

The third subsystem is the host computer interface section. This subsystem 
implements a communication link to a host computer. As discussed later, this 
communication link which can be serial or parallel will allow us to implement a 
program development environment as well as providing a data logging facility. 
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Component Selection 

For low volume production, the percentage of the development cost attributed to 
each unit is high. It is therefore important that a suitable platform be chosen such 
that the development cost is minimal. On the other hand, for high volume produc- 
tion, the percentage of the development cost attributed to each unit is very low. It 
is then advisable to use cheap components at the expense of greater development 
effort and cost. In such cases, multiple components are often integrated into a sin- 
gle chip, and firmware programs are hand optimized to achieve short code length 
and high speed of operation. Balancing the various trade-offs for a good design is 
an art as well as a science. 

The key component in the hardware design is the microprocessor. There are at 
least three considerations; namely the computational requirement, the complexity 
of the software to be developed and the type of peripherals required to interface 
to the sensors and actuators of the plant. 

To begin with, the microprocessor selected must be powerful enough to com- 
plete all the control algorithm computations required within the sampling inter- 
val. It might be thought that a good gauge of how much computational effort 
is required can be obtained by taking time needed to execute programs of simi- 
lar complexity on a general purpose computer of known power, and to take into 
account differences in the average efficiency of compiled code on the general pur- 
pose computer and that of the targeted processor. 

The second consideration is the complexity of the program to be developed. 
If the program to be developed is very complex, then it is advisable to select a 
commonly used microprocessor. This is because commonly used microproces- 
sors usually have many supporting tools, either from the manufacturer or other 
third parties. These tools greatly simplify the task of developing the program. Be 
aware that a highly optimized compiler can produce object code that runs an order 
of magnitude faster on the same hardware platform than that of a nonoptimized 
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compiler. 

The third consideration is the type of peripherals required for the microproces- 
sor. There is a class of integrated circuits, commonly known as microcontrollers 
or single chip microcomputers, which not only contain the basic microprocessor 
but also many other peripherals integrated onto the chip. Using one of these chips 
will reduce the chip count on the controller board. It will also reduce the possi- 
bility of hardware bugs as well as increase the reliability of the final board. These 
three considerations should be carefully weighed in selecting the microprocessor 
for the controller board. 

The amount of EPROM and RAM to provide for the controller system will 
very much depend on the program it is executing. Checking the memory needed 
when a similar program is compiled on a general purpose computer will generally 
give a good gauge of the memory required by the target system, qualified by 
possibly different compiler efficiencies. Typically, controller systems do not have 
huge memory requirements. Otherwise there may be a need to look into using the 
cheaper dynamic RAM in controller systems instead of static RAM. Of course 
dynamic RAM is more complex to interface to the microprocessor, and is slower 
than static RAM. 

The input/output interfaces required will very much depend on the application 
at hand. The interface requirement may tie in closely with the selection of the mi- 
croprocessor for the system. If there is a microprocessor that meets the application 
requirements, and in addition, has all the peripheral interfaces needed integrated 
on chip, then it is only natural to select that particular processor for the applica- 
tion. Otherwise it may be necessary to use separate interface components, or seek 
a solution using two or more processors. 



Pulleys 




FIGURE 3.3. Measurement of swing angle 
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Controller for an Overhead Crane 

Figure 3.2 shows the schematic of a laboratory size overhead crane. The crane is 
designed to pick up a load from one location and move it to another location. The 
bigger cousins of such cranes are used at many harbors to move containers from 
ships onto trucks and vice versa. The crane has a movable platform running on a 
pair of parallel suspended tracks. The load in turn is suspended from the platform. 
The height of the load from the ground is controllable by a motorized hoist which 
is mounted on the structural frame. The platform’s position is controlled by an- 
other motor driving a geared belt attached to the platform. The DC motors have 
quadrature phase optical encoders attached to their shafts, which allow precise 
calculation of relative positions. The swing of the load is also measured using a 
potentiometer as depicted in Figure 3.3. 

From a control point of view, there are three sensors and two actuators. The 
three sensors are the two optical encoders that produce two sets of quadrature 
phase signals and a potentiometer setup that produces a varying voltage output. 
The two actuators are the two DC motors with their speed and torque controlled 
by pulse-width modulated (PWM) input signals. 

Based on the sensor and actuator requirements, the control system must have a 
target microprocessor, two up/down counters for the two sets of quadrature phase 
signals, an analog-to-digital converter for the output of the potentiometer and two 
bidirectional PWM driving channels. 

In view of the requirements, it turns out to be possible with current technology 



Serial Communication 
Interface to Host 



To Overhead Crane 
Control Panel 

To LED Status 
Indicators 

Analog Input 

Motor 1 Control 

Motor 1 Encoder 

Motor 2 Control 

Motor 2 Encoder 




FIGURE 3.4. Design of controller for overhead 
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to select a microcontroller with all the required peripherals and a built-in serial 
port on-chip as the target processor for the controller board. The hardware design 
is shown in Figure 3.4. 

Note that the number of components used in this application is minimal. The 
processor has a built-in 8 KB of EPROM and 256 bytes of RAM. These are suffi- 
cient for implementing simple control algorithms. However for program develop- 
ment purposes as well as possibly implementation of more complex algorithm, a 
32 KB RAM chip is included. 

Controller for a Heat Exchanger 

Figure 3.5 shows the schematic diagram of a scaled down version of a heat ex- 
changer. The control objective is to maintain the level and temperature of the tank 
at some preset references. The two valves can be electronically opened or closed 




FIGURE 3.5. Schematic of heat exchanger 
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to control the amount of steam and water through the heat exchanger. 

For this plant, there are two sensors, namely the temperature and level sensors 
and two electronic valves acting as the actuators. (The pressure indicators are not 
used in this control loop.) All the sensors and actuators are of industrial grade, 
and deliver and receive 4 mA to 20 mA signals, respectively. To interface to these 
sensors and actuators, two analog-to-digital and two digital-to-analog converters 
are required. 

In view of the requirements, we select a microcontroller target processor with 
two channels consisting of analog-to-digital and digital-to-analog converters. The 
hardware design of the controller system for the heat exchange is shown in Fig- 
ure 3.6. 



Serial communication 
interface to host 




FIGURE 3.6. Design of controller for heat exchanger 



Software Development Environment 

For commonly used microcontrollers, processor emulators are provided by the 
manufacturer or some third party vendors. A processor emulator emulates all the 
functionalities of the specific microcontroller in real-time. It usually consists of an 
electronic board connected to a personal computer or workstation. The electronic 
board has a plug to be inserted into the microcontroller socket of the target con- 
troller hardware. This plug has the same pin-outs as the specific microcontroller. 
The emulator then provides a software development and debugging environment 
for the stand-alone system using the resources of the PC or workstation. 

Processor emulators are ideal for software development. However they are ei- 
ther very expensive, or only made available to developers of high volume prod- 
ucts. Low volume controller based designers often have difficulty getting such an 
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emulator. In the absence of an emulator, what then is the procedure for develop- 
ment of the control software? Traditionally, it is done as follows. 

The software that implements the control algorithm is coded on a workstation 
using a native editor of the workstation. It is then compiled using a cross-compiler 
into binary object code for the target stand-alone system. The EPROM of the 
target system is then unplugged from the stand-alone board and “erased”. This 
involves subjecting the EPROM to about 15 minutes of ultra-violet radiation, but 
recent CMOS RAM technology with a switch can simplify this process. Once the 
EPROM is “erased”, the binary object code can be programmed into the EPROM 
using an EPROM programmer. Once this is done, the EPROM can be plugged 
back onto the target board. Power to the board can then be applied to verify the 
software code. If there are bugs in the program, the whole cycle will have to be 
repeated. 

Even for a simple program, it is not unusual for the procedure to be repeated 
a number of times before all the bugs are removed. If the program to be im- 
plemented is complex, as would be the case for the algorithms proposed in this 
book, the development using this approach could be very inefficient. Note also 
that the success of this procedure is on condition that the hardware is already de- 
bugged and verified to be working properly. In the event that this is not the case, 
the procedure could be quite frustrating for the inexperienced, and even for the 
experienced. 

In this section, we present a procedure that will greatly simplify the devel- 
opment procedure. Consider the setup of Figure 3.7. The host computer is any 
workstation or personal computer where a cross-compiler for the target micropro- 
cessor of the stand-alone controller system exists. The host computer is linked to 
the controller system through either a serial link or a parallel link. 

Programs for the target microprocessor can be developed and cross-compiled 
to an object code on the host computer. Once this is done, the program can be 
sent or downloaded to the target system through the communication link. This is 
easily accomplished using a serial port driver program available with most gen- 
eral purpose host computers. To complete the loop, a program residing on the 
target computer has to be written to accept the object code when it is downloaded 
through the communication link, load it into memory and execute it when the 
entire file is received. 




FIGURE 3.7. Setup for software development environment 






254 Chapter 9. Real-time Implementation 



The procedure is conceptually simple. The question is: How easily can this 
be achieved in practice? The key difficulty in the procedure described appears 
to be the writing of the program which resides on the target system referred to 
as the bootstrap loader. This program has to be developed in the traditional way 
as described above, that is through the programming of the EPROM. Clearly, if 
this program is more complex than the controller algorithm that we are going to 
develop, then there is no advantage in adopting this procedure. Let us examine the 
complexity of writing the bootstrap loader. 

The format of the object code which the cross-compiler produces varies. Be- 
side the program data in binary format, the object code also contains information 
to load the program data. This is the address information that tells a loader where 
in the memory space the program is to be loaded. The format of the object code 
should be given in the manual of the cross-compiler. If the object code is sent 
down to the target microprocessor through the communication link, then it is the 




FIGURE 3.8. Flowchart for bootstrap loader 
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task of the bootstrap loader to interpret these codes; essentially to extract the ad- 
dress information and load the program data at the appropriate memory locations 
according to the address information extracted. 

Currently popular microprocessors are those from Intel* or Motorola^. For the 
microprocessors from these two manufacturers, the cross-compilers usually com- 
pile to the Intel Hex format and the Motorola S format, respectively. A bootstrap 
loader is but a few lines of code. 

A C-program that implements a bootstrap loader to receive a downloaded ob- 
ject code in the Intel Hex format can be written to implement the flowchart of 
Figure 3.8. In the event that the cross-compiler does not output the object code 
in the Intel Hex format, it is probably easier to write a converter program on the 
host computer to convert the foreign format to the Intel Hex format. In this case, 
all the programming support resources are available to the user. 

With this setup, the environment of the stand-alone system is virtually extended 
to include that of the host computer. All the resources of the host computer such 
as the keyboard, monitor, disk drives, editor, compiler, etc. are now accessible to 
the stand-alone system and can be used in the development of its software. Prac- 
tically, the stand-alone system is loaded with an operating system, and a Serial 
Port Operating System, (SPOS) as compared to merely the Disk Operating Sys- 
tem (DOS) of a PC. Moreover, once the program is developed and verified to be 
working correctly, it can then be recompiled and programmed into the EPROM 
so that the eventual controller system can operate as a stand-alone unit with the 
serial link removed. 

Software Debugging Environment 

In the last section, a serial link and a bootstrap loader is used to virtually extend the 
hardware and software capability of the stand-alone system. The logical next step 
is to provide a debugging facility on the stand-alone system for the development 
of its software. Of course the necessity of providing this facility will depend on 
the complexity of the software to be developed. In the case where the software 
is relatively simple, it may not be cost effective to invest time in the provision 
of the debugging facility. In the other case where the software to be developed is 
complex, the investment will save considerable time in the subsequent debugging 
of the software. 

There are at least three levels of debugging to be provided for effective devel- 
opment of a program. The first is to provide a programmer with a single stepping 
facility. This facility allows the programmer to execute one instruction at a time. 
After each instruction, the programmer is allowed to view the values of variables 
to ascertain the correctness of the program. This facility is normally used to debug 
short segments of a program which contain complex logical flows. It is usually not 
used to debug the entire program, with possibly thousands of lines, as it is too time 
consuming. 

* Intel® is a registered trademark of the Intel Corporation. 

^Motorola® is a registered trademark of the Motorola Corporation. 
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The second allows the programmer to set a breakpoint somewhere in the pro- 
gram. In this case, the program is run from the beginning until the breakpoint. 
After the breakpoint, control is transferred to a monitor routine which allows the 
programmer to view the values of the variables in the program and then engage 
the stepping facility. This facility is used to debug larger segments of a program 
which are suspected to contain logical errors. 

The third level is to provide a facility for the programmer to monitor the values 
of variables in the program as the program is executed. On a general purpose 
computer, this level is equivalent to printing the variables on the monitor screen 
as the program is executed or to a file for later reference. This level allows the 
programmer a general view into the program, being especially useful when the 
exact error in the program cannot be isolated. 

Single Stepping 

Single stepping cannot be provided entirely through software means. The mi- 
croprocessor must either have hardware features that support this operation, oth- 
erwise certain external hardware logic has to be included. To provide a single 
stepping facility, the microprocessor or external hardware logic should have the 
capability to generate an exception or interrupt after executing each instruction. 
Many processors provide such a facility and the facility is usually activated by 
setting a bit in the status register of the microprocessor. In the Intel 80x86 series 
of microprocessor, this bit is known as the trap flag whereas in the Motorola 68K 
series microprocessor, it is known as the trace flag. 

Figure 3.9 shows the mechanism that implements the single- stepping facility 
in microprocessors. To invoke single stepping, the user sets the relevant bit at the 
point where the microprocessor is required to go into the single stepping mode. 
Once this bit is set, the microprocessor will on completing the current instruction 
generate an internal interrupt. This interrupt will cause the microprocessor to do 
an indirect branch to an interrupt service routine through a fixed memory location 
in the interrupt vector table. 

The user has to write the interrupt service routine to perform the tasks desired. 
The task is usually to display or store the values of the microprocessor’s internal 
register as well as certain relevant memory locations so that the user can ascertain 
their validity. The starting address of the service routine is then inserted at the 
predetermined fixed memory location in the interrupt vector table. 

As part of the indirect interrupt call, the return address and the program status 
word are saved on the stack and the single- stepping bit in the status register is 
cleared. Once the single- stepping bit is cleared, the microprocessor will no longer 
generate an interrupt after completing each instruction. The interrupt service rou- 
tine is therefore executed without interrupt after every instruction. 

At the end of the interrupt service routine is an “interrupt return” instruction. 
When this instruction is executed, the previously saved return address and pro- 
gram status word is restored into the microprocessor. Control is thus returned to 
the main program, where the next instruction will then be executed. Of course, 
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FIGURE 3.9. Mechanism of single- stepping 



when the program status word is restored, the single- stepping bit which is part of 
the program status word is again set. The microprocessor will therefore generate 
an interrupt after executing the next instruction in the main program. The whole 
procedure is thus repeated until an instruction to clear the single- stepping bit is 
encountered in the main program. 

From the above explanation, a single- stepping facility is achieved by writing an 
interrupt service routine and inserting the start address of the routine at a prede- 
termined fixed address location. The facility can then be invoked by set a relevant 
bit in the program status word. 

Breakpoints 

Technically, breakpoints can be generated entirely using software means. The eas- 
ier way is to insert calls to a breakpoint service routine at the places where break- 
points in the main routine are desired. Once the execution of the main program 
reaches the place where the breakpoint routine is inserted, control will be trans- 
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ferred to the breakpoint service routine. In this routine, users can examine the 
various internal values of the microprocessor. Once this is completed, control can 
be returned to the main program and execution of the main program can continued 
to the next breakpoint. 

In fact breakpoints and single- stepping can be alternatively invoked in the same 
main program. This allows a localized as well as global debugging of the main 
program. 



Continuous Monitoring 

To monitor certain variables, calls to a subroutine which puts the variables con- 
cerned into temporary memory locations are inserted at the relevant places in the 
main program. The saved values of the variables can then be transmitted to the 
host PC via the serial link after execution of the main program is completed. Al- 
ternatively, the values of the variables can be saved into a first-in, first-out queue. 
The serial port can then be placed in an interrupt mode, which will transmit in 
real-time any saved variables in the queue to the host PC. The implementation of 
the software queue is depicted in Figure 3.10. 

The subroutine put char () is called whenever certain variables are to be 
saved. This subroutine saves these variable in the first-in, first-out queue imple- 
mented in RAM. The serial port is connected in the interrupt mode. Thus when- 
ever the transmit buffer in the serial port is empty, it will generate an interrupt. 
An interrupt service routine will then be called to retrieve a value from the queue 
and write it into the serial transmit buffer for onward transmission to the the host 




queue. 



FIGURE 3.10. Implementation of a software queue for the serial port 
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PC. This additional task for the microprocessor does not normally take up too 
much resources from the microprocessor. Of course, in real-time data processing 
where the main task of the microprocessor takes up all the processing time of the 
microprocessor, then this procedure is not feasible. However in many cases, the 
microprocessor is selected to have an additional 10% to 15% processing power 
than the requirement. This additional processing power is to allow for unforeseen 
additional tasks within the main task. This additional processing power if not uti- 
lized can be used to implement the debugging facilities. 

A Dual-Processor Solution for Fast Processes 

For relatively fast processes and relatively complex control algorithms, such as 
those encountered in the aerospace industry, the sampling interval and computa- 
tions required tends to be beyond the limits of simple processor technology. There 
is motivation to move to more powerful processors. However beside requiring a 
powerful processor, the controller system will also require all the interfaces to the 
sensors and actuators of the plant to have rapid data flows. 

Powerful processors optimized for number crunching such as DSPs are the 
processors needed in such applications. It is ideal if the DSPs also come 
with many peripheral functions integrated on-chip. However currently this is 
not the case. Firstly, the demand for such chips is not high enough for in- 
tegrated circuit (IC) manufacturers to invest in the design and manufacturing. 
Secondly the die size for such highly integrated chips would be large, and 
consequently the production yields of the chips low and therefore the costs 
are higher. 

To provide a hardware platform with a powerful number crunching capabil- 
ity as well as numerous interface functions, it seems appropriate to exploit the 
strength of both the DSPs and microcontrollers in what we call a dual chip 
design. We present one such design which we call the Fast Universal Con- 
troller (FUC). The high computation speed of the FUC is derived from the 
DSP and it can go up to tens or hundreds of MFLOPS (millions of floating 
point operations per seconds), depending on the particular DSP selected. It is 
universal in that it contains numerous input/output interfaces to satisfy most 
controller needs. 

The design of the FUC is shown in Figure 3.11. It can either function as a 
stand-alone unit or maintain a serial communication link to a host computer. In- 
ternally, the FUC consists of two distinct processing modules which perform dif- 
ferent tasks. The first module (DSP module) contains a DSP and associated resi- 
dent RAM. This module is assigned to perform number crunching in the overall 
strategy of the FUC. Typically, it is programmed to execute all tasks of the FUC 
unit except servicing of external devices. The DSP does not maintain any hard- 
ware link to external devices except to the second module through a DMA (direct 
memory access) channel. 

The heart of the second module (microcontroller module) is a microcontroller 
which provides interaction with all external processes or devices through its built- 
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Serial lines 




FIGURE 3.11. Design of a fast universal controller 



in I/O interfaces as well as other specialized chips such as analog-to-digital con- 
verters (ADC), digital-to-analog converter (DAC) and programmable digital I/O 
chip. Any information to be passed to the DSP section is done through the DMA 
channel. 



DSP Module 

This module consists of the DSP, in our application an AT&T* DSP32C, 128 K 
32-bit words of RAM and the necessary logic circuits to interface the RAM to the 
DSP There is no nonvolatile memories or interfaces to external devices. The only 
link to the external world is a DMA channel to the NEC§ microcontroller. The 
DMA controller is a built-in feature of the DSP This DMA controller is accessible 

^AT&T® is a registered trademark of the AT&T Corporation — formerly the American Telephone 
and Telegraph Company. 

^ NEC® is a registered trademark of the NEC Corporation — formerly the Nippon Electric Com- 
pany, Limited. 
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by the microcontroller through a 16-bit control/data bus. Through this control/data 
bus, the microcontroller is able to control the read/write of any memory location 
within the DSP memory address space. This bus also allows the microcontroller 
to do a soft reset of the DSP. 

The program to be executed on the DSP is loaded onto the DSP’s memory 
by the microcontroller through the DMA channel. Once the entire program is 
loaded into the DSP’s memory, the microcontroller will initiate a software reset 
of the DSP. This causes the DSP to branch to the first executable instruction of 
the downloaded program and commences execution of the application. 

In real-time execution of the application program, the DSP will monitor sensor 
readings and send control signals to actuators. Since there is no hardware link 
between the DSP and the external devices, interaction is done using a software 
approach as follows. Within the memories of the DSP, certain fixed memory lo- 
cations are reserved as mailboxes for each of the hardware ports. That is, there is 
a memory location that serves as a mailbox for each of the peripherals; analog-to- 
digital converters (ADC), digital-to-analog converters (DAC), pulse width modu- 
lator (PWM) ports, etc. For input devices such as the ADC, the microcontroller is 
responsible for obtaining the readings from the hardware device and then trans- 
mitting this information to its corresponding mailbox (in the DSP memory ad- 
dress space) using the DMA channel. The DSP then picks up the data from the 
mailbox (memory location). Similarly, for any data that the DSP needs to write 
to an output device (such as a DAC), it places the data into the corresponding 
mailbox for the output device. The microcontroller next retrieves the data from 
the mailbox using the DMA channel and then appropriately transmits the data to 
the actual hardware device. In this way, the DSP interfaces to external devices are 
implemented by simply accessing memory locations. Furthermore the hardware 
devices have “perceived” hardware characteristics for the DSP that are exactly the 
same as those of RAM. 

Microcontroller Module 

This module consists of the microcontroller and all the peripheral chips that are 
needed to service all the various hardware processes. The microcontroller is not 
involved in the execution of the control algorithm. However it has to perform a 
number of supporting tasks. 

The first task is to ensure that application programs for both the DSP and itself 
are properly loaded. In the stand-alone mode of the FUC unit, the program for 
the DSP is burned into the EPROM of the module. Once the unit is powered 
up, a bootstrap loader is executed to transmit the DSP program through the DMA 
channel to the proper location within the DSP memories, reset the DSP and branch 
to the beginning of the NEC microcontroller program. In the mode where the 
programs are to be downloaded from the host computer, the bootstrap monitors 
the serial link for the downloaded program. Once the program is downloaded from 
the host computer, the bootstrap loader receives the programs and loads them into 
the appropriate memory locations of the DSP or microcontroller accordingly. 
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The second task is to maintain synchronous timing for the entire FUC unit. The 
built-in timer of the microcontroller is tasked to maintain the precise time required 
to implement sampling intervals. 

The third task is to service all the peripheral devices. For input devices, the mi- 
crocontroller will have to ensure that proper timing and procedures corresponding 
to each device be adhered to so that a reliable reading is obtained. The obtained 
reading is then transmitted to the device’s mailbox within the DSP module at the 
appropriate instant. For output devices, the microcontroller will obtain the data 
value from the appropriate mailbox and then drive the corresponding hardware 
output device. All device characteristics are to be taken care of at the microcon- 
troller’s end. 

Finally should there be a need to send any data to the host computer for data 
logging, the microcontroller will appropriately condition the data and then send 
them via the serial line to the host computer. 

As as concluding remark to our discussion of a dual processor solution, it is 
interesting to note the modular approach adopted in the design. This approach 
allows an easy development of the control algorithms. There is no need for the 
control engineer to worry about servicing the peripheral devices. This is an ad- 
vantage since having to service peripheral devices in real-time always makes pro- 
gramming complicated because different devices have different operating require- 
ments. As an example of this increased complication, an ADC requires some finite 
time interval from the command to start converting to delivery of the converted 
data. This finite time interval may take many processor working cycles. In this 
case, it may not be feasible for the processor to just wait for the ADC to complete 
the conversion, because of other commitments. It is then necessary to set up an 
interrupt routine so that when the ADC is ready to deliver the data, the processor 
is interrupted to read in the data. In our case the control engineer can assume that 
all these operating requirements are taken care of by the microcontroller module 
and the data will be available in the mailbox for the peripheral when and as it is 
required. There is no need for background and foreground differentiation of vari- 
ous parts of the program, which then allows straightforward programming of the 
control algorithm. The program on the microcontroller will be servicing all the 
peripherals and takes care of all the operating requirements. This program may 
be a little more complicated but since there is one task less, the overall design is 
somewhat simplified. 

A Personal Computer-based Solution 

In this section we describe the design of a universal input/output (UIO) board that 
plugs into the I/O bus of a personal computer (PC). The board is similar to the 
microcontroller module of the FUC except that it does not have a DMA link to 
the DSP module. Instead it has a parallel interface to the I/O bus of the PC mother 
board. This board plugs directly into the expansion slot of the PC mother board 
and the peripherals on board are then accessible by the microprocessor residing 
on the PC mother board. The design of the UIO is shown in Figure 3.12. 
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Motor control 
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The program for the control algorithm is now developed and executed on the 
PC. All the resources of the PC are available to the programmer for develop- 
ment of the software. There is no need to write a bootstrap loader. There is also 
the advantage of having a cheap and well written optimized compiler for the de- 
velopment. Although we call this solution a personal computer based solution, 
there is really no need for the PC once the program for the control algorithm 
is developed. We can transform it into a stand-alone system. We only need to 
purchase a PC mother board at low cost. The object code of the program can be 
programmed into a EPROM and this replaces the EPROM that comes with the PC 
mother board. The UIO will also have to be plugged into the PC mother board. 
Once done, power can be applied and the three components consisting of the PC 
mother board, UIO and the control software (on EPROM) are transformed into 
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a stand-alone controller system. What is further required is a properly designed 
casing. 

As mentioned before, this approach is ideal for low volume production of con- 
troller systems. Because of the high volume turnover for PC mother boards, the 
price is relatively low. Also because of the high integration used in the design of 
the mother boards, they tend to be reliable. It also has the advantage of allowing 
the control engineer to develop the software on the PC. A limitation for compu- 
tationally intensive controllers is the limited bandwidth of a PC bus. However, 
there are PC/DSP arrangements which are available now which only use the PC 
for development and not for controller implementation, and in this way overcome 
the bandwidth limitations of the PC. 



Main Points of Section 

In this section, we have described the hardware platform to implement control al- 
gorithms. A methodology to simplify the development of a microprocessor based 
controller system is presented. A dual-processor design approach for plants which 
require more computational resources and a personal computer based approach 
are also described. 



9.4 Software Platform 

The software for the control algorithm can be developed on a number of platforms. 
The platform will depend on the complexity of the algorithm as well as the degree 
of optimality of the code desired. To obtain an object module that is short as well 
as executes in the shortest possible time, it is necessary in some cases to program 
in assembly language and do hand optimization of the code. This however is very 
time consuming and difficult unless the problem is relatively simple. Of course for 
the development of products that will be mass produced, this may be a worthwhile 
investment. However for most situations this is to be avoided. 

The other alternative is to code in the C language, which is frequently the soft- 
ware platform of choice. The object code produced is not as optimized as that 
produced by a hand optimized assembly language source, but it is a good com- 
promise between development efficiency and code optimality. 

Control via Coding in MATLAB ^ 

With more powerful computers and more complex problems to be solved, the as- 
sociated software is correspondingly more complex. Just as the trend in hardware 
is towards higher integration, so there is a trend for software to move in this direc- 
tion. There is now a greater dependency on prewritten software modules. In the 



^MATLAB® is a registered trademark of the MathWorks, Inc. 
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past, it would takes an experienced programmer many days or weeks to write a 
graphical based user interface, but today this can be accomplished in a few min- 
utes by any programmer using one of the many application packages. Software 
libraries are not new, however the functions offered by today’s software library 
are no longer basic. They now come in more integrated forms and each function 
accomplishes a more complex task. The question that arises is: How does this 
trend affect the coding of control algorithms? 

MATLAB was originally a public domain software package developed to pro- 
vide easy access to matrix and linear algebra manipulation tools developed by 
the UNPACK and EISPACK projects. Since then it has been further developed 
by the Math Works into a commercial version. Rewritten public domain versions 
are also available and users are advised to check the Usenet archive sites for an- 
nouncement of the latest release. MATLAB comes with a programming language 
command set as well as many built-in functions that are commonly used in design 
and simulation of control systems. The command language allows users to write 
their own functions or script files, referred to as m-files. These m-files which are 
in plain text format can be called from within the MATLAB environment to per- 
form the functions they are written to perform. The m-file functions have been 
grouped together to form toolboxes for application to tasks such as system identi- 
fication, signal processing, optimization, and control law design. There are many 
such toolboxes in both the commercial and public domain. 

The thousands of m-file functions implement all kinds of control system design 
and simulation functions. In fact many algorithms are already available as m-files. 
With the aid of these m-files, often there is no necessity to code algorithms from 
scratch. The main program to perform the task defined is merely a series of calls 
to each of these m-file functions. For illustration, rather than for application by 
the reader, the design and simulation of an LQG controller is given in Figure 4.1. 
Note that dlqr and dlqe are m-file functions that compute the controller and 
estimator gains of the LQG controller respectively. 

The question we now ask is: How can the above design and simulation program 
be used to control a real plant? Consider the personal computer based solution 
described in the last section. We mentioned that the program for the control algo- 
rithm is to reside on the PC and the input and output linkage to the plant is done 
through the universal input/output (UIO) card. In order for us to use the above 
MATLAB program to control the plant, we need to do three things. First, we need 
to install MATLAB on the PC. Second, we need some routines which allow us 
to access the UIO peripherals from within the MATLAB environment. Third, we 
need to modify the above program so that instead of simulating the plant dynamics 
to get the plant output, we can directly call a routine to read output measurement 
data from the actual plant. Similarly the controller output will have to be sent to 
the plant through the UIO. 

Two C-language programs ADC.c and DAC.c can be written. ADC.c, when 
called, accesses the analog-to-digital converter of the UIO and returns the digital 
values of the analog signal. DAC.c on the other hand accepts the parameter passed 
to it by the MATLAB program and sends it to the digital-to-analog converter of 
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% State space definition of nominal plant 
A = [0.7 0 ; 0 0.8]; 

B = [0.5 ; 0.8]; 

C = [1 1.5]; 

D = [0]; 

$ [Definition of design parameter for LQG controller 
Q_control = C *C; 

R_control = 1; 

Q_ostireator = 1; 

R_est irttaSkf® = 1 ; 

% Design of LQG controller 

K_control = dlqr (A, B, Q_control, R_control) ; 
K_estimator = A*dlqe (A, B, C, Q_estimator, R_estimator) ; 

% Initial^ at i|@f|: of variables 
xk = j[Q; 0] ; 
uk = 0; 
xhat = [0;0]; 

while ( 1 ) , 

yk = C*xk + D*uk + .i'ahd(^)i); 
xk = A*xk + B*uk + B*rand(l,l); 
uk = K_control*xhat ; 

xhat= A*xhat + B*uk + K_estimator* (yk - C*xhat); 



FIGURE 4. 1 . Program to design and simulate LQG control 



the UIO for conversion. The two programs can be linked with MATLAB which 
then allows them to be called from within MATLAB. 

With the routines just described, the simulation program in Figure 4.1, modified 
as illustrated in Figure 4.2 (again not necessarily for use by the reader), becomes 
a real-time control program. 

There are three places where modification is necessary. First, the two 
lines that simulate the plant is removed and replaced by yk = ADC ( 1 ) . 
This is really a call to ADC.c to obtain in digital form the analog output 
value of the plant. Second, an additional line DAC (uk) is added. As the 
reader would have guessed, this is to send the control uk to the actuator 
of the plant via the UIO. The third modification is the introduction of the 
statements start \_time = clock which assigns the current time to the 
variable start \_time and etime (clock, start \_time) <= sam- 
ple\_interval which gives the difference between the current time and the 
time recorded in the variable start \_time. These statements ensure that the 
while loop is executed once every sample\_interval seconds. 

Implementing the controller algorithms using m-files has another advantage. 
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% State space definition of nominal plant 
A = [0.7 0 ; 0 0.8]; 

B = [0.5 ; 0.8]; 

C = [1 1.5]; 

D = [0]; 

$ [Definition of design parameter for LQG controller 
Q_control = C' *C; 

R_control = 1; 

Q_cscireator = 1; 

R_est irttaSb!® = 1 ; 

% Design of LQG controller 

K_control = dlqr (A, B, Q_control, R_control) ; 

K_estimator = A*dlqe (A, B, C, Q_estimator, R_estimator) ; 

% Initial’i&ati&ft of variables 
xhat = [ 0 ; 0 ] ; 

% Initialization for real-time control 
sample_interval = 1; 

st<6HS_time t 5 block; 

while ( e t i rr.e ( c ock , star L_L i me ) >= sample_interval ), 
start_time = clock; 

DAC (uk) ; 
yk = ADC (1) ; 

uk = K_control*xhat ; 

xhat= A*xhat + B*uk + K_estimator* (yk - C*xhat) ; 



FIGURE 4.2. Program to implement real-time LQG control 

For most purposes, before an algorithm is implemented, it is designed and 
simulated to check that it is performing to expectation. This part is usually 
performed using a high level design and simulation package such as MAT- 
LAB. Often only after the simulation shows promising results does one de- 
cide to apply the algorithm to the real plant. The practice is then to re- 
code the algorithm in some programming language such as C for implemen- 
tation on the real plant. This step is usually done by another group of sys- 
tem programmers. However program bugs and communication between the de- 
sign group and the implementation group may lead to problems and delays in 
the project. Our proposed approach, working with m- files makes implemen- 
tation straightforward for the control designers. There are only three places 
where modifications are made. If the real-time control does not match the 
simulated results, the focus can then be on the control issues rather than the 
implementation issues. 
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A MATLAB M-file Compiler 

The approach, working with m-files described in the last section, has at least four 
drawbacks. First, MATLAB is available only on general purpose computer sys- 
tems. The approach is not possible on a computer platform not supported by MAT- 
LAB. As an example, the FUC described in the previous section is not supported 
by MATLAB and therefore the approach is not applicable. Second, the approach 
requires MATLAB to be running in order to execute the m-file program that im- 
plements the control algorithm. MATLAB is a huge program and requires con- 
siderable computer resources to run. This is wasteful since the m-file program 
may not require all the MATLAB resources. Third, the approach allows the m- 
file program to be easily altered. This is an advantage during development as it 
allows the designer to fine tune the control algorithm quickly. However once the 
system is in operation and is maintained by operators who are not familiar with 
the design of the control algorithms, easy alteration of the m-file is strongly dis- 
couraged. Fourth, MATLAB is an interpreter and has high execution overhead. To 
enhance execution speed, the obvious step is to generate compiled object code for 
the control algorithms using a MATLAB-to-C converter such as is now commer- 
cially available. Such a converter converts any m-file into its equivalence ANSI- 
compliant C source. The equivalent C source can then be compiled using a C 
cross compiler to an object code for any target processor. The object code can 
then be programmed into an EPROM to give a turnkey controller system. With 
the MATLAB-to-C converter, the four drawbacks described above are overcome. 



Main Points of Section 

In this section, we describe various ways to implement the control algorithms in 
software. The traditional way is to code in assembly language or C. We suggested 
an alternative way, to program in m-files format. This allows the use of high level 
m-file functions available in MATLAB toolboxes to be used in the development 
of the control algorithms. To facilitate the development of stand-alone systems, 
the role of a MATLAB-to-C converter that converts m-files into C sources is de- 
scribed. The C source can then be compiled using a C cross compiler into object 
code for any hardware platform. 



9.5 Other Issues 

There are a few other issues which the readers will probably encounter in the 
implementation of a control system. We will briefly highlight them here. 

Implementation of Direct Feedthrough 

Strictly speaking, a direct feedthrough is not realizable in the discrete-time imple- 
mentation of controllers on a computer. This is because the processor will have 
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to take some finite time to process any data that is read in through the input inter- 
face before it can be written back to the output interface. Thus it appears that there 
should be at least one sample delay in the controller. In other words, the controller 
must be strictly causal. However in the event that the computational time of the 
controller is small in comparison to the sampling period, a direct feedthrough can 
be approximated. In this case the control signal is sent to the actuator as soon 
as it become available. Of course, if the hardware constraints preclude a direct 
feedthrough, or an approximation to this, then the controller design should yield 
a strictly causal controller. 

Integer Representation 

With floating point microprocessors, users do not have to be concerned with over- 
flow or underflow of arithmetic operations. However, in general, it is also true that 
floating point processors are significantly more expensive than integer based pro- 
cessors. In fact in many designs, preliminary implementation is made on floating 
point based processors. Once the algorithm is shown to be working, the design is 
then converted to run on an integer based processor for mass production. 

The process to convert a floating point based program to an integer based pro- 
gram can be rather tedious. In essence, one has to examine all intermediate results 
of the program to determine the range of values they can take. If the range is be- 
yond that offered by an n-bit integer representation, then either the number of bits 
used to represent the number has to be increased or that value has to be scaled 
back accordingly. Often this is achieved by shifting the binary point to the right 
by m-bits or equivalently by dividing by a 2 m operation. Correspondingly, if the 
range is too small, then we may not be getting sufficient resolution and there is a 
need to expand the resolution by shifting the binary point m-bits to the left. 

There is also often a need to change the order in which certain computations are 
performed to avoid overflow or underflow. An example is to compute the equation 

*=X>-X> 



where a,- , £>, are positive numbers 



z = ^(a; - bi). 



Finite Word Length 

In the development of the algorithms in the book, infinite precision representation 
of numbers is assumed. This assumption is not valid in real-time implementation 
in digital computers. The numbers used in the calculations have finite word length. 
We will not do any analysis on the effects of finite word length implementations 
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of the algorithms in the book. The reader is referred to Williamson (1991) and 
Gevers and Li (1993) and also to Middleton and Goodwin (1990) for in-depth 
treatments of the various issues. However we stress that there are different sources 
of truncations which can adversely affect the performance of the algorithms. 

Main Points of Section 

In this section, three issues are flagged. The first is the implementation of direct 
feedthrough in discrete- time systems. The second is the effect of integer represen- 
tation in control algorithm implementation. The third is the effect of finite word 
length in microcomputer systems. 



9.6 Notes and References 

This chapter sets the stage for real-time implementation of the algorithms pre- 
sented in the book. The chapter begins with an introduction to discretizing of 
continuous-time plants for computer implementation. It then moves on to hard- 
ware/software platforms. Most of these techniques are known in industries or 
in laboratories, or presented in restricted publications. The material presented is 
based on our experience in implementing these systems both in universities as 
well as in industry. 




CHAPTER 10 



Laboratory Case Studies 



10.1 Introduction 

The aim in this chapter is to present some student laboratory case studies of the 
application of high performance control theory and its real time implementation. 
Also, some simulation feasibility studies are included. The case studies are of 
necessity limited and perhaps contrived to some degree since they do not arise 
from fully funded engineering research and development programs for real world 
applications. Each study is presented somewhat qualitatively in order to illustrate 
aspects of engineering design rather than to allow for complete reproducibility of 
the results or to represent a triumph of the approach. The way is opened for the 
reader to achieve any triumphs. 



1 0.2 Control of Hard-disk Drives 

Hard-disk drives are an important data-storage medium for computers and data- 
processing systems. In the hard-disk drive, rotating disks coated with a thin mag- 
netic layer or recording medium are written with data in concentric circles or 
tracks. Data is read or written with a read/write head which consists of a small 
horseshoe shaped electromagnet. This read/write head is usually driven by a ser- 
vomechanism system. Within the servomechanism, two different controllers are 
used. The task of the first controller is track seeking, that is to move the read/write 
head from track to track. Usually an optimal controller that minimizes the time 
taken to do this is used. The task of the second controller is track following, that 
is to maintain the head above a particular track while data is being read or written. 
This is a regulation problem. Currently a combination of classical control tech- 
niques, such as lead-lag compensators, PI compensators and notch filters are used 
in the track following algorithm. 
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The objective in hard-disk drive development is towards smaller drives with 
high data storage capacity and faster seek and read/write times. Track to track 
seek time is governed by the maximum deliverable power of the motor driving 
the read/write head and the mass of the read/write head. The rate of read/write 
operations is governed by the speed of the spindle motor that rotates the magnetic 
disks of the drive. 

There are two ways to achieve a smaller disk drive with higher capacity. The 
first is to reduce the size of the footprint for each bit of data. The second is to 
reduce the width of the tracks where the data is stored. The first has to do with 
magnetic storage technology, the second has to do with the quality of control of 
the read/write head above the tracks while data is read or written. At the time 
of writing, the width of each track in a disk drive is still large compared to the 
footprint of each piece of data. It appears that high rewards can be achieved by 
improving the control aspects of the drive. 

There are two primary sources of disturbance in the hard disk servo system. 
The first is a repeatable run-out (RRO). As the name implies, RRO is a periodic 
disturbance that stays locked to the disk rotation (both frequency and phase). This 
disturbance is due to imperfect or eccentric tracks. See Hara, Yamamoto, Omata 
and Nakano (1988), Chew and Tomizuka (1990) for work in this one aspect. The 
second is a nonrepeatable run-out (NRRO). NRRO is the cumulative result of 
disk drive vibrations, electrical noise in the electronic circuits and the measure- 
ment channels. Vibrations come from many sources. These sources include the 
spindle motor, spindle bearings, air movement between the head and the disk, 
the actuator and also force disturbances such as closing of covers. Unlike RRO, 
NRRO disturbances are not periodic or predictable. Hence, they are more difficult 
to reject than RRO disturbances. In this section we will present designs to reduce 
this NRRO form of disturbances. 

System Model 

Two hard disk drives are used in this case study. The first drive (Drive 1) is a com- 
mercially available 5.25 inch Winchester 2.4 GB drive with a dedicated servo sys- 
tem providing information on the relative position for the servo read/write head. 
The drive has a digital signal processor (DSP) to compute the necessary control. 
The spindle motor is rotating at 5 400 rpm and control is done at an interval of ap- 
proximately 42 jis . The existing track following regulator for the servo system is 
a combination of PI compensator, notch filter and lead-lag compensator. A block 
diagram of the servo system and its own internal controller is shown in Figure 2.1. 

The position error signal ‘pes’ is derived from the servo head representing the 
deviation of the read/write head from the required track center. Here ‘return’ is 
the output from the existing controller and ‘err_out’ is the input signal injection 
into the servo system. This servo system contains an input point, ‘stintin’, which 
is used for injecting test signals into the servo system for obtaining frequency 
response. Under normal operation of the system, this signal is set to zero, hence 
‘return’ equals ‘err_out’. 
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External controller 



FIGURE 2.1. Block diagram of servo system 

There are two possible ways to access the servo system. The first is to detach 
the existing controller and replace it with the new control algorithm. This would 
involve recoding all the functions, including hardware initialization routines cur- 
rently performed by the DSP. This is not possible without a detailed description 
of the hardware and the addresses and functionalities of all peripherals. The other 
way, which is an easy way out is to treat the servo system together with its in- 
ternal controller as the plant. An external controller is then designed to control 
this ‘plant’ . However with this approach, the external controller may not be able 
to “see” the entire spectrum of the servo system. This is because the internal 
controller may have filtered out parts of the spectrum. Nevertheless this is the ap- 
proach adopted in this case study. We will in the rest of the section consider the 
internal controller as part of the plant, termed here servo system. 

Let us model the servo system as 

(q~ l )yk = ( q~ l )uk + m ( 2 . 1 ) 

where yk = ‘pes’, Uk = ‘stimin’, ( q~ l ) = 1 + a\q~ x + • • • + a„q~ n and 

(g -1 ) = b\q~ l H 1- b n q~ n . This can be written as a linear-in-the-parameter 

model as follows. 



yk = <t>' k 0 + Wk ( 2 . 2 ) 

where 0 = [ -yk - i ■■■ -y*-» ]' and 0 = [ai ... a n bi ... b n ]'. 

Excitation signals applied to the inputs and the outputs are logged. A least 
squares algorithm is then used to estimate the parameters of the model (2.1) of 
the servo system. The magnitude of the frequency response of an estimated 18th, 
19th and 20th order model is shown in Figure 2.2, which can be compared to a 
plot using a spectrum analyzer as shown in Figure 2.3. The two figures show close 
correlation. 
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Frequency (Hz) 



FIGURE 2.2. Magnitude response of three system models 




Frequency (Hz) 



FIGURE 2.3. Measured magnitude response of the system 

The second hard disk drive (Drive 2) is an IBM* Winchester 3.5 inch disk. It 
has a rotating speed of 4 3 16 rpm and control is performed at the rate of 6.9 kHz. 
The magnitude of the frequency response of an estimated third order model for 
this drive is shown in Figure 2.4. 

An Optimal Disk Controller for Drive 1 

A second order model is used in the design of the controller for the servo system of 
Drive 1 . This model is obtained by approximating the high order models obtained 
in the previous section and is given as follows. 

y k - 1.83y*_i + 0My k - 2 = 0.001m*_i - 0.007m*- 2 + w k . (2.3) 

Qualitatively, the objective of the controller is to ensure that the deviation of the 
read/write head position from the center of each track ‘pes’, is small at all instants 
of time without saturating the actuator. This will then allow the track width to 
be set to twice the worst deviation (or slightly larger) of ‘pes’ . It is clear such 
a qualitative specification translates quantitatively into an optimal controller that 
minimizes the infinity norm of ‘pes’. 

*IBM® is a registered trademark of the International Business Machines Corporation. 
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Let us select an error signal as = [ y k iu k ] ' where A. is a constant and set up 
a performance index as 

J = Iklloo = max(||y*|loo , HAMfcHoo). (2.4) 

Using the Q parameterization approach of Chapter 2, we have a closed-loop 
system given in terms of Q as Fq. Using the notation of Chapter 4, we have the 
following optimization task. 

min fj$QU>|| . (2.5) 

QeRHoc 11 ^ 1100 

Now the solution to this optimization task depends on the characteristic of the dis- 
turbance u> into the system. In our case, w is certainly bounded. We could further 
assume w to be 2-norm bounded or infinity-norm bounded. Either assumption as 
shown in Chapter 4 leads to different optimization tasks. 



min 

QeRH^ 


Fq 2. 


if ||w|| e* 2 . 


IM| 2 <1, 


(2.6) 


min 

QeRHoc 


Veil 


if llwll e £oo. 


IMIoo<1. 


(2.7) 



In this application, as mentioned, the targeted disturbances to be rejected by the 
controller are the NRRO and they can be attributed to disk drive vibrations and 
electrical noise in the electronic circuits and the measurement channels. Therefore 
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either the 2-norm or infinity-norm bounds can be considered to be good approxi- 
mations. We thus present the results of controllers designed for both optimization 
tasks (2.6) and (2.7). 

Table 2.1 shows the amplitude and energy of ‘pes’ when a Ffr and a l\ op- 
timal controller are used to control the system. This is compared against the 
case where no external controller is used. Each run consists of 200000 sam- 
ples. In general the l\ optimal controller reduces the maximum magnitude of 
‘pes’ compared to when no external controller is used. The optimal con- 
troller on the other hand does little to reduce the maximum magnitude of ‘pes’ . 
We have also computed the power of ‘pes’ in the three cases. As expected, 
for this criterion the H 2 controller turns in the best performance, reducing the 
power of ‘pes’ by about 20% compared to when no external controller is used. 
The power of ‘pes’ is also reduced by 10-15% when an l\ optimal controller 
is used. 




‘pes’ value 




FIGURE 2.5. Histogram of ‘pes’ for a typical 
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1st Run 2nd Run 3rd Run 4th Run 


Amplitude 

(Min/Max) 


h 

Hi 

NR 


-22/22 -25/23 -23/24 -22/23 

-27/26 -27/26 -25/26 -26/25 

-24/26 -27/26 -25/26 -26/25 


Energy 


h 

H 2 

NR 


29.2 33.1 32.4 33.2 

28.0 30.7 29.3 31.0 

33.9 39.1 35.1 36.3 



TABLE 2.1. Comparison of performance of l\ and Hi controller 

To understand the results better, a histogram of a typical run is shown in Fig- 
ure 2.5. The axis shows the magnitude of ‘pes’ while the y-axis represents the 
number of occurrences N. Attention is drawn to the two extreme ends of the his- 
togram shown enlarged. Here, we observe that the l\ optimal controller not only 
reduces the maximum amplitude of ‘pes’, but also reduces the number of occur- 
rences at higher values of ‘pes’ . 

An Adaptive Controller for Drive 2 

This part of the case study is extracted from Li (1995) and Horowitz and Li (1995). 
The adaptive controller used to control Drive 2 is depicted in Figure 2.6. 

The adaptive scheme is a specialization of the adaptive techniques presented in 
Chapter 5 and Chapter 6. Assuming both the nominal plant Go and the nominal 
controller Kq to be the zero operators, then the stable coprime factorizations of 
the nominal plant and controller are trivially given by 

N = 0, M= 1, U= 0, V=l. 




FIGURE 2.6. Adaptive controller for Drive 2 
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FIGURE 2.7. Power spectrum density of the ‘pes’ — nominal and adaptive 



With these factorizations, the class of all plants parameterized by S (which is here 
the servo system G), is then given by 



G(S) = G — 



N + SV 
M + SV 



S, 



and thus S = G. 

The parameters of S are identified on-line as B/ A and the minimum variance 
controller is given by 



Q 



1 + 



QB 

A 



An input disturbance centered at about 60 Hz is injected into the system via w\ of 
Figure 2.6. 

Figure 2.7 shows the power spectrum density of the ‘pes’ obtained in the above 
experiment. With the adaptive augmentation, an improvement of about 43% over 
the nominal internal controller is recorded. Figure 2.8 shows the error rejection as 
a function of frequency. 
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FIGURE 2.8. Error rejection function — nominal and adaptive 
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Main Points of Section 

In this section, a case study on the control of two hard disk drives is described. 
Two optimal schemes are implemented, namely an l\ and an Hi optimal scheme 
as described in Chapter 4 . The results show an improvement in performance over 
the case where these controllers are not present. For the second drive, an adaptive 
controller based on the scheme described in Chapter 5 is implemented. A further 
significant improvement in performance is recorded. 



10.3 Control of a Heat Exchanger 

Heat exchangers are extensively used in many industrial process installations such 
as power plants, chemical processing plants and oil refineries. In this study we use 
a laboratory scale heat exchanger as shown in Figure 3.1. Although many times 
smaller than its industrial counterparts, it is a good representation for studying the 
types of problems associated with such plants. A schematic of the heat exchanger 
is shown in Figure 9.3.5 which is redepicted here as Figure 3.2. The objective of 
the control is to maintain the temperature and fluid level of the hot tank at some 
preset value. These two variables can be manipulated by controlling the rate of 
flow of steam and cold fluid into the tank through appropriately placed electronic 
valves. 

In this study, we will take the reader through the engineering cycle of commis- 
sioning a controller for such plants. We will first look at physical and structural 




FIGURE 3.1. Laboratory scale heat exchanger 
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boiler Legend 




FIGURE 3.2. Schematic of heat exchanger 



modeling to determine an approximate suitable representation of the plant. Once 
such a model structure is determined, parameter identification is then performed 
from data obtained through experimental trial runs. Based on the model obtained 
we design a simple LQG controller to control the plant. The results obtained from 
real-time control of the plant is then compared with simulation runs based on the 
model. In our study, simulation studies tally closely with that of the mn on the 
actual plant, giving us good confidence in the accuracy of the plant model. We 
have also included a numerical plant model such that the interested reader may 
experiment with the high performance controllers described in the book. 

Structural Modeling 

The plant contains two 0.5 meter cubic steel tanks. One of the tanks serves as a 
buffer tank where cold water is pumped via the heat exchanger into the hot tank. 
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The flow of cold water into the heat exchanger is controlled by the pneumatic 
valve EV1. The hot tank is equipped with an outlet pump PI, a differential pres- 
sure level transducer (LT2) and a platinum resistance thermometer (TT3). The 
shell and tube heat exchanger consist of many round tubes mounted in a cylindri- 
cal shell with their axes parallel to that of the shell. The heat exchanger is designed 
as a steam heated system where water from the buffer tank flows through the in- 
side of the tubes and is heated by steam flowing through the shell to supply the 
required heat. Steam required for the process is generated by a boiler. The flow of 
steam into the heat exchanger is controlled by the pneumatic valve EV2. 

To perform an accurate physical modeling of the underlying process is com- 
plex. We have therefore simplified the derivation so that a good understanding of 
the process is obtained without being burdened with unnecessary complexities. 

To obtain an understanding of the fluid level loop, we first maintain the steam 
input valve EV2 at a constant flow rate. Then the inflow rate q(t) is related to 
level h{t) in the hot tank by the equation 



q(t) = A 



gg£J 

dt 



(3.1) 



where A is the cross sectional area of the hot tank. 

As designed and also observed, the dynamics of the pneumatic valves and level 
sensors are very much faster than the process. These dynamics are therefore ig- 
nored and the transfer characteristics are represented by the DC gains, respec- 
tively, Gdc i , G dcv . The resultant transfer function from the valve EV1 to the level 
sensor LT2 is then given as 



^lC?) _ GdciGdcv n 

V(s) ~ As ' ’ 

A simplified representation of the shell and tube heat exchanger is shown in Fig- 
ure 3.3. The fluid that flows through the inner pipe at velocity v is heated by steam 
condensing outside the pipe. For simplicity let us ignore any spatial distribution 
of the steam temperature. The differential energy balance for the fluid inside the 
pipe over the volume element of length Sx is given by 

Rate of accumulation of internal energy 

= Enthalpy in — Enthalpy out + Heat transferred, (3.3) 



or symbolically, 

S [AipSxC(T - T r )] 

8t 

= vAipC(T - T r ) - vAipC ["(T + — Sx) - T r 1 + nDihi8x(T w - T), 



(3.4) 
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Symbols 

T(x,t) fluid temperature 

T w (x,t) wall temperature 

T v (t) steam temperature 

7> reference temperature for evaluating enthalpy 

p density of fluid 

C heat capacity of fluid 

p w density of metal in wall 

Aj cross-sectional area inside pipe 

A w cross-sectional area of metal wall 

Dj inside diameter of inner pipe 

D a outside diameter of pipe 

hj convective heat transfer coefficient inside pipe 

h 0 heat-transfer coefficient for condensing steam 

v fluid velocity 



FIGURE 3.3. Shell-tube heat exchanger 



and therefore 



ST 

~St 



ST T w — T 

■v— H 

8x T] 



1 _ nDjhi 
T\ Ai pC ' 



(3.5) 



Now, the energy balance equation for the metallic wall over the volume of length 
Sx is stated as follows: 



Accumulation of energy in wall 

= Heat transfer in through steam — Heat transfer out through fluid film, (3.6) 
that is, 

ST W 

A w 8xp w C w -^~ = tt D 0 h 0 Sx{T v - T w ) - jtDihiSx(T w - T), (3.7) 
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= — (7> - T w ) (T w - T ), 



T12 A w p w C w 



A w 



Taking the Laplace transform of (3.8) and (3.5) and solving them simultaneously, 
we have 

+ = (3.10) 

OX V V 



r\ Tlirmns + T12 + T22)’ 



Tl(Tl2T22S + T12 + T22) ’ 



which is an ordinary first order differential equation with boundary condition 
T ( x , s) = T (0, s) at x = 0. Solving this equation gives 

r(x,i) = r(o, S ) + (i- e -t 1 ^ (^W-r(<U)), (3.ii) 



where T (0, s) is the transform of the fluid temperature at the entrance to the heat 
exchanger and T v (s) is the transform of the steam temperature. If the tempera- 
ture of the fluid entering the pipe does not vary significantly, which holds true if 
the buffer tank is kept fairly constant, the transfer function relating the exit fluid 
temperature to the steam temperature is 



T(L,s) _b(s)/ -<4p)L' 

T v (s) a(s) V 6 ” 3 



(3.12) 



The transfer function relating the temperature of the inflow water from the outlet 
of the heat exchanger to the temperature of water in the hot tank can be derived 
from the energy balance equation 



Accumulation of total energy in St 

= input energy — output energy — energy lost to the environment, (3.13) 
giving 

PfAfh fCf 8{T{t) ~ Tr) = P f qoC f (T (L, t) - Tr) - Pf qoCf(T - 7» - 0. 

(3.14) 
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Since the flow rate of fluid through the heat exchanger is assumed constant as qo, 
then hf is a constant. Taking the Laplace transform of (3.14) and then solving 
with (3.12), we get 



T(s) _ Ks)(l-e 
Tv(s) a ( s }(fijdLL s + i) 



(3.15) 



Again assuming that the dynamics of EV2 is very much faster than the change in 
temperature of the tank, so that it can be ignored and be replaced by its DC gain 
K r jc, the relationship between the input voltage to EV2 and the temperature of 
fluid in the hot tank is given by 



T(s) K dcb{s) (l - e~^ L ) 
V{s)~ + 



(3.16) 



Parameter Identification 

To determine the sampling rate, step inputs are applied separately to both the 
pneumatic valves, EV1 and EV2, and the corresponding effects on the tempera- 
ture and level of the hot tank are determined. It is found that the temperature and 
level processes have time constants of approximately 46 and 13 minutes, respec- 
tively. A sampling period of one minute is therefore chosen to reflect a sampling 
rate about 13 times that of the faster process. 

Next, pseudo random binary sequences (PRBS) of amplitude ±1 are applied 
simultaneously to the two pneumatic valves. The corresponding temperature and 
level readings are then logged at one minute intervals. These are shown in Fig- 
ures 3.4 and 3.5. An off-line least square algorithm is then used to estimate the 
parameters of a linearized autoregressive exogenous (ARX) input model of the 
form 

(q~ l )yk = (q~ l )uk-n + ek, 

where n is number of delay samples. 

Parameters are estimated for various model orders and sample delays. The fol- 
lowing estimated transfer functions yield the smallest residual errors. 

-0.086 2z~ l 

11 ~~ 1 -0.995 5z->’ 

-0.007 6z~ l 

12 ~ 1 -0.995 5 z- 1 ’ 

_ -0.001 lz" 1 +0.012 2z~ 2 + 0.012 5z“ 3 + 0.003 2z~ 4 

21 ~~ 1 - 1.2629 z- 1 + 0.361 4z~ 2 - 0.107 8z“ 3 + 0.058z“ 4 ’ 

-0.005 5z -1 + 0.013 0z~ 2 + 0.020 z“ 3 + 0.002z“ 4 

22 “ 1 - 1.262 9z~ x + 0.361 4z~ 2 - 0.107 8z~ 3 + 0.058z“ 4 ' 
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In state space form, G is given as follows. 

'-0.1441 -0.359 7 0.339 2 

0.085 9 -0.033 9 0.0912 

A — 0 1.3505 0.811 1 

0 0 0.845 

0 0 0 
0.4116 0.255 3 

0.2946 -0.529 8 

0.448 8 -0.643 0 

-0.0363 -0.1814 
-0.996 1 -0.087 8 

0 0 0 0 0 0 0.08651 

0 0 0 0 0 0.0303 0 J’ 




-0.2646 0 

1.772 0 

-0.230 0.339 2 

0.629 8 0 

0 0.995 5 




(3.17) 

(3.18) 

(3.19) 

(3.20) 



Feedback Control 

The model obtained is used to design an LQG controller. To achieve zero steady 
state offset in the closed-loop system, integral action is included into the system 
by defining two more states as follows. 

Xn(k + 1) = X, L {k) + Kievel (k) - R\eveiref(k), (3.21) 

XTL{k + 1) = XTL(k) + Ttemp(&) — ^teinpref (^)i (3.22) 

where /?i ev eiref > 7?tempref and Kievel, K tem p are temperature and level reference inputs 
and outputs signals respectively. 




Minutes 

FIGURE 3.6. Temperature response and control effort of steam valve due to step change 
in both level and temperature reference signals 
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The resultant augmented state equation is then given a; 



0 

-0.1441 
0.085 9 
0 
0 



0 

-0.359 7 
-0.033 9 
1.3505 
0 
0 



0 

0.4116 
0.2946 
0.448 8 
-0.0363 
-0.996 1 



0 

0.255 3 
-0.529 8 
-0.643 0 
-0.1814 
-0.087 8 



0 0 0 0 
10 0 0 



0 

0.3392 
0.0912 
0.811 1 
0.845 
0 



0 0 0 0 0 0 

0 0 0 0 0 0.0303 



0.0865 

0 



0 

0.0303 
-0.2646 
1.772 
-0.230 
0.629 8 
0 



0 

0 

0 

0 

0.995 5 J 



The performance index is defined as 
1 N 

J = x XI ( x (k)Qc x (k) + u'(k)R c u(k )) , 
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FIGURE 3.7. Level response and control effort of flow valve due to step change in both 
level and temperature reference signals 
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The estimator gain is designed using a process noise covariance R v and n 
surement noise covariance R w with 



1” 0.024 4 2 




R W J' °1 


L ° 


0.003 2 J 


0 1 



The resultant controller is then implemented in real-time using MATLAB. The 
results are benchmarked against the ideal case, which is a simulation with the 
designed controller controlling the earlier model obtained. 

Figures 3.6 and 3.7 show the plots when simultaneous 0.5 V step changes are 
applied to both the level and the temperature reference signal whereas Figures 3.8 
and 3.9 show the responses to just a temperature reference signal step of 0.5 V. 




FIGURE 3.9. Control effort of steam and flow valves due to step change in temperature 
reference signal 
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10.4 Aerospace Resonance Suppression 

This study is not strictly a case study in that the controllers have not been im- 
plemented on aircraft to our knowledge. Rather, it is a feasibility study using the 
best available linear aircraft models. Simulation results are presented for wing tip 
accelerometer control of high order models of a supersonic airplane. Of particu- 
lar interest is the suppression of resonances in certain frequency bands. Similar 
results have been achieved for high speed civil transport aircraft models. We do 
not give the model or design details here, but simply show performance spectral 
plots and discuss the various design issues. 



Introduction 

Aircraft high frequency structural resonance modes can be excited in certain re- 
gions of the flight envelope. At the extremes of this envelope, such resonances 
lead to wing flutter and catastrophic failure. Because of a degree of uncertainty 
in aircraft models, such resonances are known to be extraordinarily difficult to 
suppress by active means. 

In this study, a combination of a robust controller and an adaptive scheme is 
used to control high frequency structural modes for aircraft models of 100th order 
or so. The objective is to suppress wing flexure or body bending resonances in the 
vicinity of 20 to 80 rad/s by means of aileron, or rudder control. Certainly, it is 
imperative that these modes not be excited by naive control actions. The sensors 
could be accelerometers on the wing tips or body extremities. Robust controllers 
by themselves may not achieve adequate performance over the entire range of 
situations of interest, so that there would be a role for adaptive augmentations to 
such robust controllers, at least at the extremes such as in emergency dive situa- 
tions. This study is a step to explore such a role using realistic high order aircraft 
models. 



Technical Approach 

There are three high order aircraft flying models which correspond to an aircraft 
flying at altitudes of 2000, 10000, and 30000 ft flight conditions, respectively. 
Spectral analysis indicates that the models exhibit two excessive wing flexure 
resonances which are to be suppressed. 

We select the aircraft model which corresponds to a flight condition at an alti- 
tude of 2 000 ft as the basis for the nominal plant. For this model, there are two 
high resonance peaks in its power spectral density function at frequencies 27.5 
and 56.4 rad/s respectively. It makes sense then to limit interest to the frequency 
band below 60 rad/s. 

Since the model is very high in order (107th order) and high frequency distur- 
bance responses above 60 rad/s are beyond our interest, it is reasonable to reduce 
the plant model order as low as possible consistent with obtaining a good con- 
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trailer design. A benefit is to reduce the complexity for the nominal plant con- 
troller. 

Next we design an LQG controller K for the reduced order plant, namely the 
nominal plant G. Of course, there is a quadratic performance index penalizing the 
disturbance responses e. 

Now applying the rationale of the direct adaptive- Q schemes of Chapter 6, we 
consider further an augmentation of the stabilizing controller K for G to achieve 
an augmented controller, denoted K(Q), which parameterizes all the stabilizing 
controllers for G in terms of an arbitrary proper stable transfer function Q. For 
different Q , the controller K (Q) will have different robustness and performance 
properties when applied to the nominal plant. Here we take Q to be an adaptive 
filter. 

The adaptive filter Q is included so that in any on-line adaptation to plants 
other than the nominal plant G, denoted G, the filter Q can be selected so as to 
minimize some disturbance response, perhaps a frequency shaped version of e^, 
the response penalized in the LQG design. 

In this latter case, the adaptive scheme and fixed controller work towards the 
same objective in the appropriate frequency bands. 

Model Reduction 

For a first cut model reduction, we consider the eigenvalue distribution, and re- 
move all the modes which are above the frequency 155 rad/s, being far beyond the 
frequency band of interest. This reduces the plant to 85th order. Then we discretize 
this model at the sampling time 0.03 s, and further reduce the model to 46th order 
by deleting those states which correspond to (stable) near pole/zero cancellations 
and also remove those states which are outside of the frequency band of interest. 
We select this reduced order discrete- time aircraft model as the nominal plant G. 
Other methods based on balanced realizations, see Anderson and Moore (1989), 
are not used at this stage because of possible numerical problems. 

Design of Nominal Controller K 

In order to design a nominal controller K , we proceed here with a straightforward 
LQG controller design for the nominal plant G. Since we aim at reducing the 
peaks of the power spectral density function, an LQ index is employed which 
weights states associated with the resonance peaks. We define the disturbance 
response to be e = [ *i «2 «3 ]' where e\ is the contribution of the states towards 
the first resonance mode, namely at 27.5 rad/s, e2 is the contribution towards the 
second mode at 56.4 rad/s and ej, is ej, = e\ + e2- By having different weighting 
factors on those responses, we design an LQ controller for the nominal plant. Our 
selection of the kernel of the cost index chosen is (4.5e 2 + e 2 + 10e| + 5 500m 2 ). 
These are selected by a trial and error approach. 

For the Kalman filter design, we select a stochastic disturbance input to the 
plant model which excites the resonances, details are omitted. 
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Frequency Shaping for the Disturbance Response 

Since the disturbance response e for the LQ design has significant energy over 
a wide frequency range, and yet our concern is only with its energy in a narrow 
frequency range, it may appear natural to pass it through a frequency shaping 
filter so as to focus mainly on the responses within the frequency band of inter- 
est. However, with a negligible weighting outside the frequency band of interest, 
adaptation to minimize the energy of the frequency shaped e may excite reso- 
nances outside this frequency band and sacrifice performance. Here we exploit 
the Kalman filter in the controller design to achieve a filtered disturbance response 
achieved by replacing states by their state estimates. 

Of course, an H 0 0 design approach for K is also possible, and in our experience 
is more direct, but it obscures to some extent the useful engineering insight as to 
how the controller is achieving its objectives. 



Frequency Shaping for the Residuals 

The Kalman filter residuals r and adaptive Q filter output s are filtered in an adap- 
tive Q scheme as discussed in Chapter 6. It may seem that these filters are as high 
order as the nominal plant. However, for our specific LQG design, there results a 
filter which can be reduced to as low as 4th order by just deleting unobservable 
states. 



Order of Q Selection 

The order of the adaptive controller Q directly affects the complexity of the adap- 
tive scheme. The number of the coefficients in Q determine the order of the on- 
line least squares parameter updating scheme. Another consideration to be kept 
in mind is the stability of Q, because the closed-loop system is stable in the case 
of the nominal plant G only when Q is stable. With finite impulse response (FIR) 
Q , the stability of Q is trivially satisfied with bounded gains, and there is a con- 
sequent simplification of the adaptive scheme. In our simulations, a 4th order FIR 
model of Q is employed, there being diminishing returns from increasing the or- 
der. With different sampling times, a different order could be more appropriate. 

It is also possible and in practice reasonable to include a fixed prefilter P r 
in the adaptive- Q loop to limit the frequency band of adaptive feedback signals 
and avoid exciting those frequency modes which may destabilize the closed-loop 
system. This is particularly important when there are high frequency unmodeled 
dynamics. It is clear that the inclusion of any stable prefilter in the adaptive- Q 
loop will not alter the optimization task, but only change the space of adaptive- Q 
action. It could well make sense to incorporate the prefilter P r into the state esti- 
mator, so that its output corresponds to a subset of the states. For example, in the 
case of a resonance or flutter suppression, the resonance or flutter state estimate 
could well be appropriate as an input to the adaptive- Q filter. 
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Three Flight Conditions 

As mentioned above, we have defined a nominal plant G and designed a direct 
adaptive- Q scheme along the lines described in Chapter 6. The performance of in- 
terest is chosen as the peak of the power spectral density function. The simulation 
results when applied in a number of situations are reported to see its robustness 
and performance over a range of uncertainties and disturbances. 

Nominal Condition 

First, we apply to the nominal (reduced order) plant model G the adaptive - Q 
scheme based on the LQG controller K . The power spectral density function of 
the output versus noise is shown in Figure 4.1. The response indicated by (a) is 
that of the closed-loop scheme with the off-line designed robust linear controller, 
and the response indicated by (b) is that of the direct adaptive- Q scheme. From 
the figure it is clear that the adaptive scheme improves the performance, namely 
reduces the peaks of the power spectral density function of the output from the 
noise by approximately 30%, at the expense of boosting higher frequency modes. 
This improvement is achieved after the nominal controller has reduced the peaks 
by an order of magnitude. 

Flight at 2 000 ft 

As mentioned before, the nominal plant is a reduced order plant based on the 
aircraft model 2 000, namely the model corresponding to the flight condition at 
altitude of 2 000 ft. When we directly employ the adaptive scheme for the nom- 
inal plant to the full order model, all the high frequency resonances are acti- 
vated and appear in the adaptive- Q loop. Recall that we can insert a prefilter in 
the adaptive- Q loop to limit the feedback of the residuals in only the frequency 
bands of interest and avoid exciting high frequency unmodeled dynamics. With a 
prefilter included, the comparison between the adaptive scheme and the closed- 
loop responses for the model 2 000, not shown here, tells us that performance is 
marginally improved by the adaptive scheme at the first resonance peak, and kept 
the same as that of the nonadaptive but robust scheme at high frequencies above 
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FIGURE 4.1. Comparative performance at 2 000 ft 
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60 rad/s. The prefilter employed in the adaptive scheme is a 16th order low pass 
filter. 



Flight at 10000 ft 

The nominal plant is now replaced by a different plant, viz. model 10 000, which 
corresponds to the flight condition at an altitude of 10000 ft. The nominal con- 
troller K, and the adaptive- Q scheme with prefilter are here taken to be the same 
as for the model 2 000. The performance of the adaptive control scheme is shown 
in Figure 4.2 as compared to that for the nonadaptive case. The adaptive scheme 
improves the performance of the closed-loop systems in this case but the improve- 
ment is very limited. 
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FIGURE 4.2. Comparative performance at 10000 ft 



Flight at 30 000 ft 

The nominal plant is here replaced by a different plant again, viz. model 30 000, 
which corresponds to the flight condition at altitude of 30 000 ft. Again the nom- 
inal controller K and the adaptive scheme setup are unchanged. In this case the 
adaptive scheme does not deliver improvement which assures us of the quality of 
the robust design. 

Remarks. 

1. The off-line LQG controller for the nominal plant G is to us unexpectedly 
robust for the three plants: model 2000, model 10000 and 30000. In fact 
the robustness-performance trade off is such that the potential for improve- 
ment via added adaptive loops can not be dramatic. On the other hand, 
should the controller not be suitably robust, performance enhancement by 
adaptive techniques could be futile. Furthermore, should there be a dra- 
matic improvement due to the adaptive- Q loop, it would be important to 
question whether there should be an improved robust design so as to reduce 
this improvement to a more appropriate level. 
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2. The simulation results for the adaptive scheme are encouraging in that they 
demonstrate that resonance suppression can occur based on on-line process- 
ing. The adaptive results do not represent a dramatic improvement over an 
off-line robust LQG design, although a more dramatic improvement is not 
precluded for other flight conditions, or variations on the adaptive scheme. 
Our first objective has been a conservative one, rather than to achieve spec- 
tacular results which may be constructed as “lucky”, as in earlier flutter 
suppression studies Moore, Hotz and Gangsaas (1982). It appears that our 
conservative objectives have been achieved. 

3. With increases to the gain on the filtered disturbance response, or without a 
prefilter in the adaptive- Q loop to limit the frequency band of residuals fed 
back through the Q loop, the adaptive scheme can be made to destabilize. 
This is expected since there can only be local stability results in the presence 
of significant unmodeled dynamics, especially at high frequency. 

4. The algorithm as studied in the simulation is impractical to implement be- 
cause of the high order. Essentially the same results are achieved working 
with reduced order filters so as to achieve a more practical design. 

5. In performing the simulations, the first 1 000 iterations are run with the only 

controller being K. During this period the least squares covariance matrix 
Pk is being updated. Then the loop is closed with zero initial condition on 
the After a further 100 iterations the adaptive Q{z~ l ) has virtually 

“converged”. For 10 000 iterations, a power spectral density measurement is 
taken. No attempt has been made at this stage to track time- varying plants. 

6. Our approach has been applied to models of completely different aircraft 
with different resonance suppression problems, namely, body bending res- 
onances rather than wing flexure resonances. Similar results seem to be 
achieved. To illustrate, results for two flight conditions of a transport air- 
craft model are presented in Figure 4.3, Figure 4.4. Here the open-loop 
resonances are shown since they are merely a factor of three above those 
under active control. 



Flutter Suppression 

In order to illustrate that dramatic performance improvement can be achieved in 
an adaptive- Q approach, we present simulation results from Moore, Hotz and 
Gangsaas (1982). The unstable flutter results from a wing bending mode and tor- 
sion mode coming together at the flutter frequency and one mode becoming un- 
stable. This will happen to any wing at sufficiently high speed, termed the flutter 
speed. 

In this study of a 65 th order flexible wing aircraft model, the adaptive- Q filter is 
driven from the flutter state estimate. Indirect adaptive- Q techniques are applied. 




10.4. Aerospace Resonance Suppression 295 







a — 














4 \ — 


Open-loop 








\ 




\ . 


Robust Co 


itroller 






\ 






Adaptive- Q 


1 scheme 






\ V 




XrC 










!_v= 






fffc 









0 10 20 30 40 50 60 70 

Frequency (Rads/sec) 



FIGURE 4.3. Comparisons for nominal model 
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FIGURE 4.4. Comparisons for a different flight condition than for the nominal case 



In particular, a second order model uncertainty is identified looking at the control 
input (wing tip aileron) prior to its entry to the estimator and the flutter state 
estimate (from wing tip accelerometers). An adaptive - Q filter is applied to assign 
the closed-loop poles to stable locations at the flutter frequency. The degree of 
assigned stability is not set to be “too” large so as to avoid excessive control 
actions which could excite lightly damped modes. Figure 4.5 shows that a flutter 
instability is controlled effectively in a few cycles, before the wing “falls off”. 
Indeed there is demonstrated in the simulations “180° phase margin” in a region 
of loop gain greater than unity! 



Main Points of Section 

The simulation results for resonance suppression at this stage are encouraging in 
that the off-line designed fixed LQG controller gives robust performance, and the 
adaptive- Q scheme is seen to only improve the performance further. Of course 
some engineering is required to achieve robust LQG designs, associated pre- 
filters, and adjustment law gains to achieve such success. In some situations the 
adaptive- Q methodology can achieve dramatic results by achieving “180° phase 
margins”, as in the case of flutter suppression. 
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FIGURE 4.5. Flutter suppression via indirect adaptive- Q pole assignment 



10.5 Notes and References 



Resonance Suppression 

Our first studies in this topic are reported in Moore, Hotz and Gangsaas (1982) 
where an indirect adaptive- Q approach is used to suppress catastrophic wing flut- 
ter. Indeed, this study provided many of the insights used subsequently for devel- 
oping the theory of the adaptive- Q approach. The Q filter in this study achieved 
a weakened form of minimum variance control. Later studies sought to achieve 
adaptive- Q filters based on LQG and pole assignment indices(Chakravarty and 
Moore, 1985; Chakravarty and Moore, 1986). Subsequently, less ambitious stud- 
ies in resonance suppression of lightly damped modes using direct adaptive- Q 
techniques were employed, thereby checking the validity of this approach, see 
also Moore, Xia and Xia (1989). Other resonance suppression studies have helped 
develop our understanding, namely Irlicht, Mareels and Moore (1993) and Telford 
and Moore (1990). 





APPENDIX A 



Linear Algebra 



This appendix summarizes the key results of matrix theory and linear algebra 
results used in this text. For more complete treatments, see Barnett (1971) and 
Bellman (1970). 



A. 1 Matrices and Vectors 

Let and denote the fields of real numbers and complex numbers, respectively. 
The set of integers is denoted ={1,2,...}. 

An n x in matrix is an array of n rows and m columns of elements xij for 
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*1 m 
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The matrix is termed square when n — m. A column n-vector x is an n x 1 matrix. 
The set of all n -vectors (row or column) with real arbitrary entries, denoted ", is 
called «- space. With complex entries, the set is denoted " and is called complex 
n-space. The term scalar denotes the elements of , or . The set of real or 
complex nxm matrices is denoted by nym or " xm , respectively. The transpose 
of an n x m matrix X, denoted X', is the m x n matrix X' = ( xji ). When X — 
X', the square matrix is termed symmetric. When X — —X', then the matrix is 
skew symmetric. Let us denote the complex conjugate transpose of a matrix X 
as X* — X'. Then matrices X with X = X* are termed Hermitian and with 
X = —X* are termed skew Hermitian. The direct sum , of two square matrices 
X. Y, denoted X + Y, is [ q y ] where 0 denotes a zero matrix of appropriate 
dimensions consisting of zero elements. 
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A.2 Addition and Multiplication of Matrices 

Consider matrices X, Y e " xm or ” xm , and scalars k,£ e or . Then Z = 
kX + £Y is defined by Zij = kxij + tyij. Thus X + Y — Y + X and addition 
is commutative. Also, Z = AT is defined for X an n x p matrix and T an 
p x m matrix by Z = (zij), Zij = X!it=i x ik}’kj, and is an n x m matrix. Thus 
W = XYZ = (XY)Z = X(YZ) and multiplication is associative. Note that 
when XY — Y X , which is not always the case, we say that X, Y are commuting 
matrices. 

When X'X — 1 and X is real then X is termed an orthogonal matrix and 
when X*X — I with X complex it is termed a unitary matrix. Note real vectors 
x, y are orthogonal if x'y = 0 and complex vectors are orthogonal if x*y = 0. 
A permutation matrix n has exactly one unity element in each row and column 
and zeros elsewhere. Every permutation matrix jt is orthogonal. Ann x n square 
matrix X with only diagonal elements and all other elements zero is termed a 
diagonal matrix and is written X — diagtxi i , X22 . . . , x nn ). When xp — 1 for 
all i and x,y = 0 for i / j, then X is termed an identity matrix, and is denoted 
or just I. Thus for an n x m matrix T, then YI m — Y — I„Y. A sign matrix 
S is a diagonal matrix with diagonal elements +1 or —1. Every sign matrix is 
orthogonal. 

For X e mx " and Ye pxr denote by X <g> T the matrix of dimension 
mp x rn, a block matrix having as i y'th block the matrix x,y Y . 



A.3 Determinant and Rank of a Matrix 

A recursive definition of the determinant of a square n x n matrix X, denoted 
det(X), is 

det(X) = y + j Xij det (X u ), 

j= l 

where det(A/ ; ) denotes the determinant of the submatrix of X constructed by 
deleting the i th row and the j th column. The determinant of a scalar x is the scalar 
x itself. The element ( — l) 1-1- - 7 det(A i; ) is termed the cofactor of x,y . The square 
matrix X is said to be a singular matrix if det(X) = 0, and a nonsingular matrix 
otherwise. It can be proved that for square matrices det (XT) = det(X) det(T). 

For A € nx P, Y e pxn we have det( /„ + A T) = det(/ p + TA).In particular 
with p = 1, for x, y e n det (/„ + AT') = 1 + Y'X. 

The rank of an n x m matrix A, denoted by rk(X) or rank(A), is the maximum 
positive integer r such that some r xr submatrix of A, obtained by deleting rows 
and columns is nonsingular. Equivalently, the rank is the maximum number of 
linearly independent rows and columns of A. If r is either m or n then A is full 
rank. It is readily seen that 

rank(A) + rank(T) —m< rank(AT) < min(rank(X), rank(T)). 
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A.4 Range Space, Kernel and Inverses 

For an n x m matrix X , the range space or the image space ( X ), also denoted 
by Im(X), is the set of vectors Xy where y ranges over the set of all m vectors. Its 
dimension is equal to the rank of X. The kernel ker(X) of X is the set of vectors 
z for which Xz = 0. It can be seen that for real matrices, (X') is orthogonal to 
ker(X), or equivalently, with yi = X'y for some y and if Xyi = 0, then y' x y 2 = 0. 

For a square nonsingular matrix X, there exists a unique inverse of X, denoted 
X -1 , such that X -1 X = XX -1 = I. The i/th element of X -1 is given from 
det(X) -1 x cofactor of xp. Thus (X” 1 )' = (X') -1 and (XT) -1 = 7 -1 X -1 
where the inverses exist. More generally, a unique (Moore-Penrose) pseu do-in- 
verse of X, denoted X # , is defined by the characterizing properties X # Xy = y 
for all y € (X') and X # y = 0 for all y e ker(X'). Thus if det(X) ^ 0 then 

x # = X -1 , if X = 0, X # = 0, (X # ) # = X, X # XX # = X # , XX # X = X. 

For a nonsingular n x n matrix X, a nonsingular pyp matrix A and an n x p 
matrix B, then provided inverses exist, the Matrix Inversion Lemma states 

(7 + XBA~ 1 B , )~ 1 X = (X -1 + BA~ l B')~ l 

= X - XB(B'XB + A)~ l B'X , 
and 

(7 + XBA~ l B')- l XBA~ l = (X -1 + BA~ 1 B , )~ l BA~ l 
= XB{B’XB + A) -1 . 



A.5 Eigenvalues, Eigenvectors and Trace 

For a square n x n matrix X, the characteristic polynomial of X is det(z7 — X) 
and its real or complex zeros are the eigenvalues of X, denoted A,- . The spectrum 
spec(X) of X is the set of its eigenvalues. The Cayley -Hamilton Theorem tells 
us that X satisfies its own characteristic equation with det(z7 — X) = p(z), 
j (X) = 0. For eigenvalues A,-, then Xu, = A ,u; for some nonzero real or complex 
vector Vi, termed an eigenvector. The real or complex vector space of such vectors 
is termed the eigenspace. If A,- is not a repeated eigenvalue, then u ; - is unique to 
within a scalar factor. When X is diagonal then X = diag(Ai, A 2 , . . . , A„). Also, 
det(X) = n" =1 A ( - so that det(X) = 0, if and only if at least one eigenvalue is 
zero. As detfz I — XY) = det(z7 — 7X), XY has the same nonzero eigenvalues 
as YX. 

A symmetric, or Hermitian, matrix has only real eigenvalues, a skew symmet- 
ric, or skew-Hermitian, matrix has only imaginary eigenvalues, and an orthogonal, 
or unitary matrix has unity magnitude eigenvalues. 

The trace of X, denoted tr(X), is the sum YH= 1 x a = Xj=i Notice that 
tr(X + T) = tr(X) + tr(7), and with XY square, then tr(XT) = tr(TX). Also, 
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tr(X'X) = V" V" x, 2 ,. and tr 2 (XT) < tr(X'X) tr(y'y). A useful identity is 

det(A) = *»(*>. 7 



A.6 Similar Matrices 

Two n x n matrices X, Y are called similar if there exists a nonsingular T such 
that y = T~ l XT . Thus X is s im ilar to X. Also, if X is similar to Y, then Y 
is similar to X. Moreover, if X is similar to Y and if Y is similar to Z, then X 
is similar to Z. Indeed, similarity of matrices is an equivalence relation. Similar 
matrices have the same eigenvalues. 

If for a given X, there exists a similarity transformation T such that 
A = T~ l XT is diagonal, then X is termed diagonalizable and A = 
diag(A.i, A.2 , . ... k„) where Xj are the eigenvalues of X. The columns of T are 
then the eigenvectors of X. All matrices with distinct eigenvalues are diagonal- 
izable, as are orthogonal, symmetric, skew symmetric, unitary, Hermitian, and 
skew Hermitian matrices. In fact, if X is symmetric, it can be diagonalized by 
a real orthogonal matrix and when unitary, Hermitian, or skew-Hermitian, it can 
be diagonalized by a unitary matrix. If X is Hermitian and T is any invertible 
transformation, then Sylvester’s Inertia Theorem asserts that T* XT has the same 
number P of positive eigenvalues and the same number N of negative eigenvalues 
as X. The difference S = P — N is called the signature of X, denoted sig(X). 



A.7 Positive Definite Matrices and Matrix 
Decompositions 

With X = X' and real, then X is positive definite (positive semidefinite or non- 
negative definite) if and only if the scalar x'Xx > 0 (x'Xx > 0) for all nonzero 
vectors x. The notation X > 0 (X > 0) is used. In fact X > 0 (X > 0) if and 
only if all eigenvalues are positive (nonnegative). If X = YY' then X > 0 and 
YY' > 0 if and only if Y is an m x n matrix with m < n and rk Y = m. If Y = Y 1 , 
so that X = y 2 , then Y is unique and is termed the symmetric square root of X, 
denoted X 1 / 2 . If X > 0, then X 1 / 2 exists. 

If y is lower triangular with positive diagonal entries, and YY' — X, then Y is 
termed a Cholesky factor of X. A successive row by row generation of the nonzero 
entries of Y is termed a Cholesky decomposition. A subsequent step is to form 
y A y' = X where A is diagonal positive definite, and Y is lower triangular with 
Is on the diagonal. The above decomposition also applies to Hermitian matrices 
with the obvious generalizations. 

For X a real n x n matrix, then there exists a polar decomposition X = &P 
where P is positive semidefinite symmetric and © is orthogonal satisfying ©'© = 
©©' = /„. While P = (X'X) 1 / 2 is uniquely determined, © is uniquely deter- 
mined only if X is nonsingular. 
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The singular values of possibly complex rectangular matrices X , denoted 
( X ), are the positive square roots of the eigenvalues of X* X . There exist unitary 
matrices U, V such that 



a\ 0 ... 0 

0 : 



V'XU 



0 



0 



0 



0 o n 



=: £. 



0 



0 0 



If unitary U, V yield V'XU, a diagonal matrix with nonnegative entries, then 
the diagonal entries are the singular values of X. Also, X = VHU' is termed a 
singular value decomposition (SVD) of X. 

Every real m x n matrix A of rank r has a factorization A = XT by real 
m x r and r x n matrices X and Y with rkX = rkT = r. With X e mxr 
and Y e rx ", then the pair (X, T) belong to the product space mxr x rx ". 
If (X, T), (Xi, Xi) e mxr x rxn are two full rank factorizations of A, i.e. 
A = XT = XiXi, then there exists a unique invertible rx r matrix T with 
(X,T) = (XiT-i.TTi). 

For X a real n x n matrix, the QR decomposition is X = OR where © is 
orthogonal and R is upper triangular (zero elements below the diagonal) with 
nonnegative entries on the diagonal. If X is invertible then © , R are uniquely 
determined. 



A. 8 Norms of Vectors and Matrices 

The norm of a vector x, written ||x||, is any positive valued function satisfying 
11*11 > 0 for all x, with equality if and only if x = 0, || ,vx || = |s| ||x|| for any scalar 
s, and ||x + y|| < ||x|| + ||y|| for all x, y. The Euclidean norm or the 2-norm is 
||x || = xf) 1 / 2 , and satisfies the Schwartz inequality |x'y| < ||x|| ||y||, with 

equality if and only if y = sx for some scalar s. Other norms are ||x = max |x; | 

and ||x|| ! = £"=i |x/|. 

The induced norm of a matrix X with respect to a given vector norm is de- 
fined as || X || = maxtofpj ||Xx||. Corresponding to the Euclidean norm is the 
2-norm ||X|| 2 = t-mlAX' X), being the largest singular value of X. Correspond- 
ing to the vector norms ||x , ||x|| ] there are induced matrix norms 1 1 X 1 1^ = 
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max,- YTj= l \ x ij | an( l II Hi = maxj x i.i I • The Frobenius norm is || X|| F = 

tr^X'X). The subscript F is deleted when it is clear that the Frobenius norm 
is intended. For all induced norms and also for the Frobenius norm ||Xx|| < 
l|X|| lull. Also, ||X + F|| < ||X|| + ||F|| and ||XF|| < ||X|| ||F||. Note that 
tr(XF) < || X ||/7 1| Y\\p. The condition number of a nonsingular matrix X rela- 
tive to a norm || • || is ||X|| ||X — 1 1| . 



A.9 Differentiation and Integration 



Suppose X is a matrix valued function of the scalar variable t. Then X (t ) is called 
differentiable if each entry xij (t) is differentiable. Also, 



dX = / dxj j \ 
dt \ dt ) ’ 



Also, f Xdt = (/ Xijdt). Now with 0 a scalar function of a matrix X, then 



— = the matrix with i jth entry . 

3X 3 xij 

If is a matrix function of a matrix X, then 

3<J> 3 (pa 

— = a block matrix with i /th block — - . 

3X J 3X 

The case when X, <t> are vectors is just a specialization of the above definitions. If 
X is square (n x n) and nonsingular, (d/‘dX)(tr(W X-')) = -X -1 WX -1 . Also 
log det(X) < tr X — n and with equality if and only if X = /„. Furthermore, if X 
is a function of time, then (d/dt)X~ l (t) = —X~ 1 (dX/dt)X~ l , which follows 
from differentiating XX -1 = I. 

If P — P', (3/3 x){x'Px) = 2 Px. 



A. 1 0 Lemma of Lyapunov 

If A , B, C are known n x n , m x m and n x m matrices, then the linear equation 
AX + XB + C = 0, has a unique solution for an n x m matrix X if and only if 
Xi (A) + Xj ( B ) ^ 0 for any i and j. In fact [/ <g> A + B' ® I] vec(X) = - vec(C) 
and the eigenvalues of [I <g> A + B' <g> /] are precisely given by k,(A) + Xj(B). 
Here vec(X) stands for the column vector obtained from the matrix X by stacking 
the columns of X from left to right under one another in the vector vec(X). If 
C > 0 and A = B' , the Lemma of Lyapunov for AX + XB + C = 0 states that 
X = X' > 0 if and only if all eigenvalues of B have negative real parts. 

The linear equation X — AXB = C, or equivalently, [/„ 2 — B' <g> A] vec(X) = 
vec(C) has a unique solution if and only if Xi ( A)Xj (B) 1 for any i, j . If A = B' 
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and |A.,-(A)| < 1 for all i, then for X — AXB = C, the Lemma of Lyapunov states 
that X — X' > 0 for all C = C' > 0. 

Actually, the condition C > 0 in the lemma can be relaxed to requiring for 
any D such that DD' = C that (A, D) be completely controllable, or (D, A) be 
completely detectable, see definitions Appendix B. 



A. 1 1 Vector Spaces and Subspaces 

Let us restrict to the real field (or complex field ), and recall the spaces ” 
(or "). These are in fact special cases of vector spaces over (or ) with the 
vector additions and scalar multiplications properties for its elements spelled out 
in Section A.2. They are denoted real (or complex) vector spaces. Any space over 
an arbitrary field which has the same properties is in fact a vector space V. For 
example, the set of all m x n matrices with entries in the field as (or ), is a 
vector space. This space is denoted by mxn (or mxn ). 

A subspace W of V is a vector space which is a subset of the vector space V. 
The set of all linear combinations of vectors from a nonempty subset S of V, de- 
noted L(S), is a subspace (the smallest such) of V containing S. The space L(S) 
is termed the subspace spanned or generated by S. With the empty set denoted 
then L((p ) = {0}. The rows (columns) of a matrix X viewed as row (column) vec- 
tors span what is termed the row (column) space of X denoted here by [X] r ([X] c ). 
Of course, (X') = [X] r and (X) = [X] c . 

The orthogonal complement of a subspace W of V is denoted as W ± . It is a 
subspace of V, consisting of all vectors v e V such that v'w = 0 for all we W. 
For a matrix X we have (X') 1 - = ker(X). For two subspaces W. U e V we 
denote by W © U the space spanned by any combination of W and U, i.e. z e 
W © t/ implies that z = w + u for some w e W and u e U . For a square matrix 
X, dimension n x n we have that ker(X) 1 - © (X) = ". 



A. 12 Basis and Dimension 

A vector space V is n-dimensional (dim V = n) if there exists linearly indepen- 
dent vectors, the basis vectors, {e \ , ei , . ■ ■ ,e n ) which span V. A basis for a vector 
space is nonunique, yet every basis of V has the same number n of elements. A 
subspace W of V has the property dim W < n, and if dim W = n, then W = V. 
The dimension of the row (column) space of a matrix X is the row (column) rank 
of X. The row and column ranks are equal and are in fact the rank of X. The 
coordinates of a vector x in V with respect to a basis are the (unique) tuple of co- 
efficients of a linear combination of the basis vectors that generate x. Thus with 
x = JA diCi, then a\, U2, . ... a„ are the coordinates. 

For a square n x n matrix X over we have that dimker(X) + dim (X) = n. 
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A. 1 3 Mappings and Linear Mappings 

For A, B arbitrary sets, suppose that for each a & A there is assigned a single 
element /(a) of B. The collection / of such is called a function, or map and is 
denoted / : A — > B. The domain of the mapping is A, the codomain is B. For 
subsets A s , B s , of A, B then f(A s ) = {/(a) : a e A s ) is the image of A s , and 
f~ l (B s ) = {a e A : f(a) e B s } is the preimage or fiber of B s . If B s = {b} is a 
singleton set we also write f~ l {b) instead of / _1 ({&}). Also, /(A) is the image 
or range of /. The notation x i-> fix) is used to denote the image f (x) of an 
arbitrary element x e A. 

The composition of mappings / : A — >• B and g : B —> C, denoted gof, is 
an associative operation. The identity map id,\ : A — »■ A is the map defined by 
a i->- a for all a & A. 

A mapping / : A — > B is one-to-one or injective if different elements of A 
have distinct images, i.e. if a\ f aj => /(«i) / fiflf). The mapping is onto or 
surjective if every b € Bis the image of at least one a € A. A bijective mapping 
is one-to-one and onto (surjective and injective). If / : A — »■ B and g : B — > A 
are maps with go f = idA, then / is injective and g is surjective. 

For vector spaces V, W over or (denoted ) a mapping F : V — »■ IT is a 
linear mapping if F(v + w) = F(v) + F(w) for any v, w € V, and as F(kv ) = 
kF(v) for any k e and any v e V. Of course F(0) = 0. A linear mapping is 
called an isomorphism if it is bijective. The vector spaces V, W are isomorphic if 
there is an isomorphism of V onto IT. A linear mapping F : V —> U is called 
singular if it is not an isomorphism. 

For F : V —> U, a linear mapping, the image of F is the set Im(F) = {u e U \ 
F(v) = u for some v e V}. The kernel of F is ker(F) = {u e V \ F(u ) = 0}. In 
fact for finite dimensional spaces dim V = dim ker(F) + dimlm(F). 

Linear operators or transformations are linear mappings T : V — > V, i.e. from 
V to itself. 

The dual vector space V* of a -vector space V is defined as the -vector 
space of all -linear maps k : V —*■ . It is denoted by V* = Hom(V, ). 




APPENDIX B 



Dynamical Systems 



This appendix summarizes the key results of both linear and nonlinear dynamical 
systems theory required as a background in this text. For more complete treat- 
ments, see Irwin (1980), Isidori (1989), Kailath (1980) and Sontag (1990). 



B . 1 Linear Dynamical S y stems 

State equations 

Discrete- time linear, finite-dimensional, dynamical systems fork = ko, ko+1, • • • 
with initial state Xk 0 e "are described by 

Xk + 1 = A k Xk + B k u k , y k = C k Xk + D k Uk, (1.1) 

where x k £ ", u k £ m , y k £ p and A k , B k , C k , D k are matrices of appropri- 
ate dimension, possibly time varying. The solution for k > ko is, 
ft- 1 

x k = ^k,k 0 x k 0 + Q>k-uBiUi. 

i=k 0 

where <S>k 0 ,k 0 = I\ ®k+i,k 0 = A k ®k,k 0 or <& k ,k 0 = A k -\ . . . A ko for k > k 0 . In 
the time invariant case <&k,ko = A k ~ k °, k > ko. 

A discrete-time, time- invariant system (1.1) is called stable, if the eigenval- 
ues of A are all located in the open unit disc {z e \z\ < 1}. This implies 
hm^oo A k = 0. 

Transfer functions 

The Z-transform of a sequence {hk \ k £ o) is the formal power series in z -1 
H(z) = f^hkz-f 




306 Appendix B. Dynamical Systems 



In discrete-time, the Z-transform yields the transfer function for the system (1.1) 
H(z) = C(zl — A)~ 1 B + D, 

in terms of the Z-transform variable z. For xq = 0, then Y (z) = (C(zJ — A)~ ] B + 
D)U (z) expresses the relation between the Z-transforms of the sequences {m*} 
and {yk}. The transfer function z _1 corresponds to a unit delay. 

A (matrix) transfer function for the system (1.1) with a state space realization 
given via the matrices (A, B,C, D) is represented as 



H : 



A B 
C D 



Continuous-time linear systems 

In continuous time we have 



dx 

~dt 



= A(t)x + B(t)u, 



y = C{t)x + D(t)u. 



( 1 . 2 ) 



The transition matrix 4>(t, to) satisfies 4>(to, to) = I and 4>(t, to) = A(t)4>(t, to). 
It has the semigroup property that 4>(t2, ti)4>(ti, to) = < f ) (t2, to). The solution of 
(1.2) starting at time to in xq is given by: 

x(t) = <t>(f, t 0 )x 0 + J 4>(t, x)B{T)u{x)dx. 

In the time invariant case A(t) — A, we have <t>(f , to) = e A( - f ~ t0 \ A time invariant 
linear system (1.2) is called stable if the real part of the eigenvalues of A are 
negative. This implies lim^oo e At — 0. 



Controllability and Stabilizability 

In the time-invariant, continuous-time, case, the pair (A, B) with A e nxn and 
B e " xm is termed completely controllable (or more simply controllable ) if one 
of the following equivalent conditions holds: 

• There exists a control u taking x = Ax + Bu from arbitrary state xo to 
another arbitrary state x\, in finite time. 

• rank {B , AB , . . . , A n ~ x B) = n. 

• {XI — A, B) has full rank for all (complex) X. 

• W C (T) = / 0 r e tA BB' e tA dt > 0 for all T > 0. 
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• AX = XA and XB = 0 implies X = 0. 

• w'e tA B = 0 for all t implies w = 0. 

• w'A‘ B = 0 for all i implies w = 0. 

• There exists a K of appropriate dimension such that A + BK has arbitrary 
eigenvalues. 

• There exists no coordinate basis change such that 



[An 


A 12 






A22J ’ 


M 



The symmetric matrix W c (T) = e tA BB'e tA dt > 0 is the controllability 
Gramian , associated with x = Ax + Bu.lt can be found as the solution at time 
T of W c (t) = AW c (t ) + W c (t)A' + BB' initialized by W c (0) = 0. If A has 
only eigenvalues with negative real part, in short Re A. (A) < 0, then Wc(oo) = 
limf-s .00 W c (t) exists. 

In the time- varying case, only the first definition of controllability applies. It is 
equivalent to requiring that the Gramian 

/ t+T 

<b(t + T, T)B{x)B'{T)<S>'{t + T, x)dx 

be positive definite for all t and some T. The concept of uniform complete control- 
lability requires that W c (t, T ) be uniformly bounded above and below from zero 
for all t and some T > 0. This condition ensures that a bounded energy control 
can take an arbitrary state vector x to zero in an interval [t, t + T] for arbitrary t. 
A uniformly controllable system has the property that a bounded K (t) exists such 
that x = ( A(t ) + B(t)K(t))x has an arbitrary degree of (exponential) stability. 

The discrete-time controllability conditions, Gramians, etcetera are analogous 
to the continuous-time definitions and results. In particular, the /V -controllability 
Gramian of a discrete-time system is defined by 

N 

\vW) := Y^, Ak BB'{A') k , 

k=0 



for N € . The pair (A, B) is controllable if and only if wl : N> is positive definite 

for all A > n — 1 . If A has all its eigenvalues in the open unit disc [z e | | z | < 1 }, 

in short |A(A)| < 1, then 



1T C := Y^,A k BB'{A') k 

k = 0 



exists and is positive definite if and only if (A, B) is controllable. 
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Observability and Detectability 

The pair (A, C) has observability/detectability properties according to the con- 
trollability/stabilizability properties of the pair (A', C')\ likewise for the time- 
varying and uniform observability cases. The observability Gramians are known 
as duals of the controllability Gramians, e.g. in the continuous-time case 

W 0 (T) := J e tA 'c’Ce tA dt, 1T 0 := e ,A ' C'Ce tA dt, 

and in the discrete-time case, 

N oo 

W o (A0 := ^(A') fc C'CA*, 1T 0 := ^(A'^C'CA*. 

k = o k=o 



Minimality 

The state space systems of (1.1) and (1.2) denoted by the triple (A, B, C ) are 
termed minimal realizations, in the time-invariant case, when (A, B) is com- 
pletely controllable and (A, C) is completely observable. The McMillan degree 
of the transfer functions H(s) = C(sl — A)~ l B or H(z) = C(zl — A)~ l B is 
the state space dimension of a minimal realization. Kalman’s Realization The- 
orem asserts that any p x m rational matrix function H (s ) with H{ oo) = 0 
(that is H{s) is strictly proper) has a minimal realization (A, B. C) such that 
H{s) = C (si — A) -1 B holds. Moreover, given two minimal realizations de- 
noted (Ai , B\,C\) and (A2, 62, C2). then there always exists a unique nonsingu- 
lar transformation matrix T such that T A\T~ l = A2, TB\ — #2, CiT -1 = C2. 
All minimal realizations of a transfer function have the same dimension. 

Balanced Realizations 

For a stable system (A, B, C ), a realization in which the Gramians are equal and 
diagonal as 



W C = W 0 = diag(CTi, . . . , a n ) 

is termed a diagonally balanced realization. For a minimal realization (A, B, C), 
the singular values Oi are all positive. For a nonminimal realization of McMillan 
degree m < n, then er m+! = 0 for i > 0. Corresponding definitions and results 
apply for Gramians defined on finite intervals T. Also, when the controllability 
and observability Gramians are equal but not necessarily diagonal, the realizations 
are termed balanced. Such realizations are unique only to within orthogonal basis 
transformations. 

Balanced truncation is where a system (A, B,C) with A e " x ", B e " xm , 
C € pxn is approximated by an rth order system with r < n as follows: As- 
suming an ordering er,- > cr !+ i, for all i the last (n — r) rows of (A, B) and last 
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( n—r ) columns of [ ^ ] of a balanced realization are set to zero to form a reduced 
rth order realization (A r , B r , C r ) € ryr x rym x pyr . A theorem of Pernebo 
and Silverman states that if (A, B, C) is balanced and minimal, and ay > ay+i, 
then the reduced rth order realization (A r , B r , C r ) is also balanced and minimal. 



B.2 Norms, Spaces and Stability Concepts 

The key stability concept for our purposes is that of bounded-input stability (BIBO 
stability). In the case of linear systems this is equivalent to asymptotic stability of 
the state space descriptions. 

To be precise, and focusing on discrete-time signals and systems, we work first 
with signals which are simply functions which map the integers to ". The set 
of signals is S = {/ : "}. The size of a signal is measured by some norm. 

Let us consider a 2-norm over an infinite horizon as 

/ °° \ 1/2 

11/112= ’ 

where ||x|| = vVx is the Euclidean norm. 

The Lebesque 2-space is defined by 

l 2 (-00, 00) = {/ e S : II /II 2 < oo} . 

Variations i 2 (0, oo), l 2 (— oo, 0) are defined similarly. When the time interval is 
understood, then the space is referred to as the l 2 space. 

The space l 2 is a Hilbert space with inner product 



</,£) = J2 8 'kf k ’ 



satisfying |(/,g)| < ||/|| 2 \\h\\ 2 and ||/||^ = (/, /). 

In the Z-domain, the 2-norm is, which follows from Parseval’s Theorem 

f i r dz} l/2 

Il/(z)ll2 = sup — (f> f-(z)f(z)-\ , 

U>0 2™ J\z\=\+€ Z \ 

where f~(z) = The Hardy 2-space H 2 is defined as 

Hi = {/(z) is analytic in |z| > 1 and ||/|| 2 < oo} . 



For any / e Hj, the boundary function 
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exists for almost all |z| = 1 ( Fatou’s theorem) and WfbWi — II /II 2 so that 

! 1 C d,7 1 

I ' 

The mapping / e H2 to fo e £2 is linear, injective and norm preserving. 

Consider now linear time-invariant systems in discrete-time with matrix trans- 
fer function H(z) : I2 h- 
The space too is defined from 

too = {H : II II oo < 00}, 
where the l 0 0 norm is defined from 

Halloo = sup rr ma x (H (z)) . 

Izl=l 

Here <7 max (-) denotes the maximum singular value. 

Stable Systems 

Stable systems H(z) are such that H (z)w(z) e H2 whenever w(z) e Hj. A 
necessary condition is that H(z) is analytic in \z\ > 1. 

The Hoo space is defined from 

Hoo = \H{z) : H{z) is analytic in |z| > 1 and ll^lh-gn < 00} - 
where the norm is an H 0 0 norm 

WWl-gn = SUP SUp <T max (H(z)) , 

e>0 |z|=l+e 

being but a mild version of the loo norm, frequently termed an H 0 0 norm and 
written Hfljloo. 

Now H(z) defines a stable system if and only if H (z) £ H 0 Q . 

In the case that H (z) e Hoo is rational then we use the notation H (z) £ RH 0 0 . 
H(z) e RHoo if and only if H(z) has no pole in |z| > 1. 

Corresponding definitions and results apply as before for continuous-time sig- 
nals and systems with Yl-00 an d $ z |=i ' s replaced by and f s= j m ■ Also f~ 

replaced by /* where f*(s) = f'(s*) where s* is the conjugate of s. There are 
additional technical issues concerning signals which differ only on (Lebesque) 
measure zero. Just to state explicitly one result, we have that H(s) e RH 0 Q if and 
only if H(s ) has no pole in Re(s) > 0. 



B.3 Nonlinear Systems Stability 

We first summarize the basic results from stability theory for ordinary differential 
equations on ", and subsequently consider corresponding results for difference 
equations. 
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Consider 



* = /(*), xe ". (3.1) 

Let / be a smooth vector field on ". We assume that /( 0) = 0, so that x = 0 is 
an equilibrium point of (3.1). Let D C " be a compact neighborhood of 0 in ". 

A Lyapunov function of (3.1) on D is a smooth function V : D —> having 
the properties 

1. V(0) = 0, V(x) > 0 for all x ^ 0 in D. 

2. For any solution x(t) of (3.1) with x(0) € D, 

V(x(f)) = ^V(x(0) < 0. (3.2) 

at 

Also, V : D —> is called a strict Lyapunov function if the strict inequality holds 

V(x(t)) = —V(x(t)) < 0 for x(t) & D — {0}. (3.3) 

at 

Theorem 3.1 (Stability). If there exists a Lyapunov function V : D —> defined 
on some compact neighborhood of 0 e ", then x = 0 is a stable equilibrium 
point. 

Theorem 3.2 (Asymptotic Stability). If there exists a strict Lyapunov function 
V : D —> defined on some compact neighborhood of 0 e ", then x = 0 is an 
asymptotically stable equilibrium point. 

Theorem 3.3 (Global Asymptotic Stability). If there exists a proper map V : 
" — > which is a strict Lyapunov function with D = ", then x = 0 is globally 

asymptotically stable. 

Here properness of V : " —> is equivalent to V (x) —> oo for ||x|| — > oc. 
Theorem 3.4 (Exponential Asymptotic Stability). If in Theorem 3.2 one has 

«i IUII 2 < V(x) < ct 2 ||x || 2 and —013 \\x \\ 2 < V(x) < — ot\ \\x \\ 2 for some posi- 
tive i = 1, • • ■ , 4, then x = 0 is exponentially asymptotically stable. 

Consider nonlinear systems with external inputs u as 
x = f(x,u), 

then BIBO stability is as for linear systems namely, that bounded inputs lead to 
bounded signals. We refer to this stability as too BIBO stability. 

Similarly for discrete time systems 

**+1 = /(x*); "• (3.4) 

Here / is a continuous map of "to ". Assume that /( 0) = 0, so that x = 0 is 
a trivial solution of (3.4). A solution of (3.4) starting in xo is denoted by xj(x 0 ). 

Let D be a compact neighborhood of /( 0) in ".A Lyapunov function V : 
D — »■ + is a continuous map such that 
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1. V(0) = 0, V(x) > 0 forx ^ 0. 

2. for any solution Xfc(xo) of (3.4) with xq £ D 

y (x*+i(x 0 )) < V (x*(x 0 )) . 

V is a strict Lyapunov function if 

3. for any solution x^(xo) of (3.4) with x 0 £ D 

V (x*+i(x 0 )) < V (x*(xo)) ; if Jt t (xo) / 0. 

The above Theorems 3.1, 3.2 and 3.3, for stability, asymptotic stability and 
global asymptotic stability, then also hold for the system (3.4). 

The trivial solution of (3.4) is called exponentially stable provided its lineariza- 
tion is stable. The linearization of (3.4) is given by 

Zk + 1 = Df(0)z k , 

where Df( 0) is the Jacobian of / evaluated at 0. The Jacobian is defined as the 
matrix of partial derivatives: 

Df(z) =: (Dfijiz)) = (^- (z)) ■ 
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Averaging Analysis For 
Adaptive Systems 

C.l Introduction 

We present here in a nutshell some ideas from averaging analysis which is a 
powerful technique to study systems whose dynamics split naturally over differ- 
ent time scales. No proofs are provided, we refer the reader to, e.g. Mareels and 
Polderman (1996). 



C.2 Averaging 

Averaging in its basic form is concerned with systems of the form: 

x k +\ =x k + iifk(x k ); x 0 ; £ = 0,1,.... (2.1) 

The parameter p is a small positive constant that characterizes the time scale sep- 
aration between the variation of the state variable x over time and the time varia- 
tions in the driving term f k (■). With time scale separation we mean the following. 
Assume for the moment that ||/jfc(x)|| < F. On a time interval of length N, a 
solution of (2.1), say x, can at most change by an amount \\x k — x k \\ < pNF, 
for \k — l\ < N. On the same time interval \\fk(x k ) — fl(xt)\\ < 2 F. The ratio 
of change between x and f k (-) is therefore of magnitude /x; the time variations 
of the state x being (potentially) ji times slower than the time variations in /. It 
is this time scale separation that hints at replacing the time- varying f k (-) by the 
time invariant averaged driving term 

^ ^ = /“(A 

k= 1 



( 2 . 2 ) 
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provided of course that the latter exists. 

In adaptive systems the time scale separation is less explicit than portrayed by 
(2.1). Part of the analysis will be precisely concerned with transforming an adap- 
tive system into the above format (2.1), at least approximately, such that standard 
averaging results can be applied. 

More precisely, we consider adaptive systems of the general form: 

x k +i = A(e k )x k + B k (0 k ), xq, 

&k+i=0k + p.gk(0k,Xk), 0 o . 

The adapted parameter vector is 9 k- The positive parameter p scales the adap- 
tation gain. We assume n to be small, it expresses explicitly that the adaptation 
mechanism progresses slowly. The rest of the state vector jt* contains mixed time 
scale behavior. Partly it contains the fast time variations due to the driving func- 
tions Bk(-), partly it contains the effect of the slowly varying 9k via the functions 
A($) and B(6). The time variations in B are typically due to external signals, 
such as reference signals, disturbances and or plant variations. It will be shown 
that under very mild assumptions the zero adaptation situation can be used as 
an approximation for the slow adaptation case. This in turn will enable standard 
averaging techniques to be used to analyze the behavior of the adaptive system. 

The methodology we are about to discuss, consisting of a zero adaptation ap- 
proximation followed by an averaging analysis, is applicable to a large class of 
adaptive systems operating under a wide variety of assumptions, not necessarily 
requiring that the model class encompasses the actual plant dynamics. 

We now introduce and discuss the basic averaging technique. 

Some Notation and Preliminaries 

In order not to overload the expressions we introduce some notation and defini- 
tions. We often need to estimate functional dependence on p. This is done via 
so-called order functions (Sanders and Verhulst, 1985): 

Definition. A scalar valued function 8(p) is called an order function if it is 
positive valued and continuous on an interval (0, p*) for some p* > 0 and 
lim^-s.0 8(p) exists, perhaps oo. 

Order functions can be defined in a more general sense. However, as we mainly 
need to compare functions in terms of orders of p and are only interested in small 
p, the above, more restrictive definition suffices. 

Example. The terms p, sin(/x), ^/Jl and 1 / p are order functions. The function 
sin ( 1 / p) + 1 is not an order function. 

The size or order of order functions can be compared as follows: 

Definition. Let <5i (p) and 82 (p) be two order functions. Then <5i (p) is said to be 
of order of 82(12), denoted as <5 1 (p) = O (82(11)), if there exists positive constants 
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H* and C such that 



<5 x (/x) < C8 2 (h) for all /x £ [0, /x*). (2.4) 



If <5i (/x) = 0(82(11)) and 82(11) = 0(8i(n)) then we say that 61 and 82 are 
equivalent order functions. 

Definition. Let <5i(/r) and 82(H) be two order functions. <$1 (/x) is said to be of 
small order of 82(H), denoted as 8 \(h) = 0(82(11)) if 



it (M) _ 



(2.5) 



Example. Now /x is o(l), as indeed lim M ^o H = 0. and obviously /x < 1 for 
all /x e [0, 1], However, sin (/x) is 0(h) on /x £ [0, n). Also /x is 0(sin(/x)) on 
/x £ [0, 7r/2). Hence /x and sin(/x) are equivalent order functions. 

Functions that do not only depend on /x can also be compared with order func- 
tions, using the following conventions: 

Definition. Let / : + x -> ", (/x , k) — »• fk(H) be continuous in /x. Let 

8 (/x) be an order function. We say that / is of order 8, denoted fk (/x) = 0(8 (/x)), 
if there exist positive constants /x* and C such that 

||/*(At)|| < C8(ji) for all k and /x£[0,/x*). (2.6) 

Definition. Let / : + x X " (/x, k, x) — > f(ji, k, x) be uniformly 

(in k) continuous in /x, x on a set [0, /x c ) x D *. We say that / is order 5 for 
some order function 8(h) if there exist a compact domain D' c D and positive 
constants /x* < He and C such that 



\\f(H,k,x)\\ < C8 (h) forall^, /x£[0,/x*), xeD'. (2.7) 



Example. Now h sin(fc) = O(h) and Vl + /xx — 1 is also 0(h)- Indeed for all 
\x\ < 1 and h e [0, 1], one has Vl + — 1 < 0.5 H- 

We will want to estimate approximation errors over a particular time interval 
that increases as /x becomes smaller. In this respect the following convention is 
useful: 

Definition. Let / : + x x ” -*■ ", (/x, k, x) /'(/x, /c, x) be uniformly 

(in k) continuous in /x, x on a domain [0, /x c ) x D. Let 8 \(h) and 82(H) be two 
order functions. We say that /(/x, k, x) is of order <$i (/x) on a time scale 82(H) on 
the set D c " provided that for any integer L there exist positive constants /x* 
and C such that for all /x £ [0, /x*) and for all x £ D 



\\f(H,k,x)\\ < C 81(H) for all k £ [0, L8 2 (h)1 (2.8) 



*g(x, k) is uniformly (in k) continuous in x on a domain D if for all e > 0 there is a S >0 such 
that for all k and all x, y e D, we have that \x — y \ <8 implies that | g(x, k) — g(y, k) \ < e. 
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Example. Let A: e , |jc| < 1 and p e [0, 1). 

x 3 sin(pk) = 0(p) on time scale 0(1), 



x 3 sin (ptk) =0(1) 

lik = O(^fp) 

(l + /x*) fc = 0(1) 



on time scale O 
on time scale O 
on time scale O 



G)- 

&)• 

G)- 



The last statement can be derived under the given restrictions for p and 
the following inequalities: 0 < 1 + /xx < 1 + p < e 1 ' and thus (1 + pt) k 
considering the time interval k £ [0, 1 / p) we get (1 + px) k < e. 



(2.9) 



x from 

< 



When discussing solutions of a difference equation such as x k +\ = x k + 
pfk(x k ) a solution is denoted by x(k, xq, ko, p) to indicate a solution that at time 
k 0 equals xo. The parameter p is included in the argument to make explicit the 
dependence on this parameter. Where no confusion can arise the shorthand x k will 
be used, and if the function / is time invariant the notation x(k, xq, p) or x k used. 



Finite Horizon Averaging Result 

We are now in a position to formulate an approximation result valid on a timescale 
1/p,. The result is stated under weak regularity conditions and in a format to 
facilitate its application to adaptive systems. 

Consider the following nonlinear and time dependent difference equation in 
standard form: 

x k+i = x k + p,fk(x k ); k € , x(0) = x 0 . (2.10) 

The parameter p is to be thought of as a small positive constant. 

We want to approximate the solution x*(xo, ko, p) of (2. 10) by some x" (xo, p) 
solution of x" + ] = x 1 / + pf a {x %), where f a is a suitable average of /. The 
approximation error, x — x a , should be o(l) on a time scale 1 / p. 

The following regularity properties are assumed: 

Assumption 2.1. Consider the difference equation (2.10), / : x ” — > " is 

locally bounded and locally Lipschitz continuous in x uniformly in k. That is, for 
each compact subset D c ", there exist positive constants Fp> and a.d possibly 
dependent on D, but independent ofk, leading to the following three properties; 

1. f is locally uniformly bounded 

F d > 0: for all x & D, for all k : ||/*(x)|| < F D . (2.11) 

2. / is locally uniformly Lipschitz continuous 

X D : for all x,y £ D, for all k : \\f k {x) - f k (y)\\ < X D ||x - y|| . 

( 2 . 12 ) 
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3. f has a well defined average, denoted f a , in that for all x e D the follow- 
ing limit exists: 

1 N 

f a (x)=lim° — ^/*(x). (2.13) 



Before continuing with our main result we offer the following observations on 
the notion of average. 

Remarks. 



1. The average f a is also Lipschitz continuous with the same Lipschitz con- 
stant a/) and locally bounded with constant Fd in the domain D as /. 
Often the average f a will have better continuity properties than the / from 
which it is derived. This may be illustrated with fk(x) — sign(sin(k) + x); 
f is not continuous but f a (x) is Lipschitz continuous in x € (— 1 , 1). 



/“(*) = 



1 

^ arc sin (x) 



1 > x > -1, 
-1 > x. 



(2.14) 



It is a nontrivial exercise to demonstrate that the limit 
1 N 

lim — 7 sign (sin (k) + x) 
JV-s-oo N f — ' 



indeed equals the above expression (2.14). It relies on the fact that the 
points k (mod 2)n are in some sense uniformly distributed over the in- 
terval [0, 27r). 

2. In the literature see, e.g. Sanders and Verhulst (1985), one sometimes 
speaks of / satisfying Assumption 2.1, Property 3 as a KBM function, 
because of the contributions to averaging theory made by the researchers 
Krylov, Boguliobov and Mitropolski. 

3. In the following situations the existence of an average is guaranteed : 

• Any function fk(x) that converges to a function independent of k, 

i.e. fk(x) = g(x) has an average given by this limit, i.e. 

f a (x) = g(x). 

• Any k-periodic function fk(x) = fk+K(x) (with period K ) has an 
average given by f a (x) = (\/K) £f =1 fk(x). 

• fk (x ) is a k-almost periodic function uniformly in x if for all e > 0 
there exists a K (e) > 0 such that for all k, x \\fk(x) — fk+K(x)\\ < e. 
Here K (e) is called an e-almost period. Any finite sum of sinusoidal 
functions is an almost periodic function, e.g. sin(k) is an almost peri- 
odic function and so is sin(k)+cos(7rk). Any almost periodic function 
has an average. 
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The above particular cases do not exhaustively describe all functions for 
which there exists an average, e.g. sin(V&) has an average, but does not 
belong to any of the above categories. 

The following result is the basic averaging result we are going to exploit. 
Theorem 2.2. Consider (2. 10). Let D c "be compact, L e and e > 0. 
Define 8(n) as: 

S(ji)= sup sup Jf l[Mx)-f a (x)]\. (2.15) 

xeD k€[0,L/n] “5 II 

Suppose that Assumption 2.1 holds. Then 8(/i) = o(l). Moreover, the solution 
xk(x o,0, aO of (2.10): 

x k+\ =x k + nfk(x k ); k e , x(0) = x 0 , (2.16) 

can be approximated by x^(xq, /x) the solution of 

x k + 1 = x k+ df a i x k^ k e > x<, (°) = *0 • (2.17) 

Furthermore, for any xo e D such that inf x6 g(/)) ||xo — x|| > e (3 (D) denotes 
the boundary of the domain D), there exists a positive constant p*(D, e, L) such 
that for all pt e [0, p*) and all k e [0, L/p\ the approximation error is 

jfj&k(x 0 , 0, /x) - x^(x 0 , /x) | = 0(j8(ji)). (2.18) 

Remark. Under a strengthened continuity assumption, assuming that fk (x ) (and 
hence also f a (x)) has a uniformly in k Lipschitz continuous partial derivative 
with respect to x, it is possible to show that the approximation error is 0(8(p)) 
rather than 0(^/8(p)). 

Infinite Horizon Result 

Again we consider a system of the form 

x k+i = x k + ufk( x kY, ke , (2.19) 

the solutions of which we want to approximate by solutions of 

4 + i = 4 + /*/*(**); k e • (2-20) 

In the previous subsection a finite horizon O (1/p.) averaging approximation result 
was introduced. 

In this section we pursue an averaging approximation result valid on the whole 
time axis under the additional condition that the averaged difference equation 
has a stable and uniformly attracting solution within the domain of interest. We 
discuss one such result. 

First we need to strengthen the notion of an average. As infinite horizon results 
are envisaged, it is natural to expect that the average /“(•) is in some sense a 
uniform over time good approximation for the time-varying function /*(•): 




C.2. Averaging 319 



Definition. The function / : " x — >• " has a uniform average f a : " — > " 

on a compact domain D c " if, for all x e D, ko, e > 0 there exists an /V > 0 
independent of ko such that for all M > N 

-| J2 (/iW-/“W) |< e - (2-2!) 



Remarks. 

1 . It follows from the above definition that for all integers L > 0 the averaging 
error 



<5*(a 0 := sup sup sup /u. V (/;-(x) - / a (x)) (2.22) 

&o>0 xeD ke[0,L/n] II i=ko II 

is an o(l) order function, i.e. lim^^o <5* Ox) = 0. 

2. The existence of an average is not sufficient to guarantee the existence of 
a uniform average. In the important situation that ^] i=0 (/,'(x) — g(x)) is 
a bounded function of k then g is a uniform average. Notice that there can 
be at most one such function g. The ^-periodic functions belong to this 
class. Also ^-almost periodic functions possess a uniform average, yet do 
not necessarily satisfy the condition that Y^!i=o(f> 00 — f° (x)) is a bounded 
function of k. 

Without loss of generality we assume the equilibrium to be the origin. More 
precisely we assume: 

Assumption 2.3. Consider the difference equation (2.20). Let f a (0) = 0. As- 
sume that there exist a neighborhood of 0, T c ", on which a positive def- 
inite, Lipschitz continuous V : T — > + and a positive definite continuous 

IV : P — > + are defined; furthermore there exist constants ji s > 0 and c > 0 

such that for all x e U := {x V(x ) < c], and all ji e [0, /x s ] there holds: 
V(x + /x/ a (x)) - V(x) < -nW(x). 

Remark. In Assumption 2.3, the set U is a compact subset of the domain of 
attraction of the equilibrium. The domain of attraction of an equilibrium is the set 
of all initials conditions for which the trajectories converge to this equilibrium. 

In order to establish the infinite horizon result we require besides the existence 
of a uniform average, that the averaged difference equation possesses a uniformly 
asymptotically stable equilibrium. For definitions of the stability concepts and 
some related results, we refer to Appendix B. 

We have the following result: 

Theorem 2.4. Consider (2.19) and the averaged equation (2.20). Let f satisfy 
Assumption 2.1 and have a uniform average f a . Let the origin be a uniformly 
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asymptotically stable equilibrium for the averaged equation (2.20) in the sense of 
Assumption 2.3. 

Let D be a compact subset of the domain of attraction. Let E be an interior ^ 
subset of D such that trajectories of (2.20) starting in E reach the set U, specified 
in Assumption 2.3, in at most O ( 1 / p.) time. 

There exists a positive constant pt* such that for all /x e [0, /x*) the solutions 
xk (x(), ko, /x) of the difference equation (2. 19) for any ko > 0 and for any xo £ E 
can be uniformly approximated by x£ (xo, /x), the solution of the averaged differ- 
ence equation (2.20) onk > ko. That is, 

\\xk(xo,ko, ff) - x£(xo, /x) | = o(l), for all k>ko- (2.23) 

Moreover, if the equilibrium is locally exponentially stable ( in that the matrix 
Df a (0) has only eigenvalues with negative real part), then the approximation 
error can be estimated as O (v / 5*(/x)), an o(l) order function. 

Essentially the theorem states that provided the averaged equation has a uni- 
formly stable equilibrium then the approximation error between the original and 
the approximate solution remains small over the complete trajectory for all those 
trajectories inside a substantial subset of the domain of attraction. 

Remark. As observed in a remark following Theorem 2.2, provided / has a 
Lipschitz continuous partial derivative with respect to x, and provided the origin is 
locally exponentially stable, the approximation error estimate can be strengthened 
to 



\\x k (x 0 , k 0 ) - xj'(x 0 ) | = 0(S*(ii)). (2.24) 



C.3 Transforming an adaptive system into standard 
form 

Recall the adaptive system (2.3) 

Xk u = A{6k)xk + Bk(6k), xo, q 

0k + 1 =0k + P-gk (fik , Xk), do. 

The system (3.1) is not in a format directly suited for averaging. Obviously 9 
is a slow variable and it is on this equation we would like to use the averaging 
ideas. However, a direct application of averaging is not possible as Xk depends on 
0k ■ In order to be able to apply averaging it is essential that we can express the 
dependence of x on 9. This is the main aim of this section. 

First we introduce some hypotheses concerning the smoothness of the adaptive 
system (3.1): 

^ E is called an interior subset of D if E c D and dD n dE = 0 
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Assumption 3.1. Let 0C m be compact. Consider the difference equation 

(3.1): 

LA: m — > pxp is continuously differentiable with respect to 0 e 0. 

2. B : m x — >• p is continuously differentiable with respect to 0 e 0. 

3. B is bounded in k. 

4. D e B k (0) is bounded in k. 

5. g : m x p x —> m is locally bounded and locally Lipschitz continuous 
in (6>, x) e 0 x X c m x p uniformly in k. 

We also require that there exist parameter values 6 such that the transition ma- 
trix A is stable. This is not an unreasonable request. If there were no such 6, then 
it is highly likely that due to the slow adaptation the x component would become 
extremely large. Also without such an assumption we have no hope that the adap- 
tation could ever stop, if 6 would converge and A (9) were unstable then x would 
diverge. 

We make this more precise: 

Assumption 3.2. There exists r > 1 such that 

S :={0 e©| A'(9)P(9)A(9) + L = P{9) with I < P(9) < rl} (3.2) 
is nonempty. 

Assumption 3.2 states that there is a nonempty subset S of © such that for 
9 e S the matrix A (0) is a stability matrix whose eigenvalues remain bounded 
away from the unit circle. (See also Section A. 10.) 

In order to describe how the slow time scale effects of 9 are observed in x we 
introduce the frozen system : 

je^- 1 (v) = A(v)jc^(v) + B*(v), x°(0, v) = x 0 , A: = 0,1,.... (3.3) 

Here xo equals the initial condition of the x state in the adaptive system descrip- 
tion (3.1). v e 0 is a fixed parameter. 

It is not difficult to demonstrate that the solutions of the difference equation 
(3.3) are for all v e S bounded and differentiable with respect to v. The following 
lemma makes this precise. 

Lemma 3.3. Consider the frozen system equation (3.3). Under Assumptions 3.1 
and 3.2 it follows that for all v e S: 

1. x®(v) is bounded uniformly in v e S; for some Co > 0: 

1 

I*t(v)| < sTr(P k llxoll + 



(3.4) 
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2. x k (v) is continuously differentiable with respect to v e S. 

3. D v x k (v) is bounded uniformly in v e S; for some C\, C 2 > 0: 

I D vX%(v) I < CiC 2 i^ + C 1 ^ : llxoll . (3.5) 

Remark. The relevance of (3.3) can be appreciated by viewing it as the descrip- 
tion of the adaptive system (3.1) where the adaptation has been switched off, i.e. 
I_i = 0. It has been associated with the names as frozen system or no adaptation 
approximation. 

The following result establishes how x in (3.1) depends on the slow variable 9 
up to terms of order of pi. 

Theorem 3.4. Consider the difference equation (3.1) under Assumptions 3.1 
and 3.2. Consider also the frozen system (3.3). 

Let (x k ,9 k ) denote the solution of the difference equation (3.1) starting in 
(xo, 9(>) at time kp. Consider 0o e S. Let x k (v) denote the solution of the frozen 
system (3.3) starting in xo at the same initial time kp. 

There exists a positive constant pi 0 > 0 such that for all pt e [0, pt 0) on the 
time interval \k : k > kp and Op e S } we have that 

1. x k (9 k ) is an O(pi) approximation ofxp: 

||xfc — x®(0*)|| < C x pc, some C x > 0. (3.6) 

2. Op can be approximated by 9® up to O (pi) on a time scale 0(1 /pi) where 
9 k is the solution of the difference equation: 

9 ° k+ 1 = 9° k + ptgkOl x° k (9° k ))- 0° = 9 ko , (3.7) 

with 

||^t - e“|| < C e pi\ some Co > 0. (3.8) 

3. Xk — x k (9 k ) = 0(/i) on a time scale 0(1 /pi). 

Remarks. 

1. Theorem 3.4 essentially allows us to decouple the x equation from the 9 
equation in the adaptive system (3.1). It allows us to study separately a fam- 
ily of linear systems (3.3) and a nonlinear time- varying equation (3.7) in or- 
der to find an approximate solution to the complete adaptive system. More- 
over the nonlinear time-varying equation governing the 0° update, equation 
(3.7), is in standard form for the application of the averaging results. This 
implies that we can further simplify the equations. This will be pursued in 
the next section. 
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2. Theorem 3.4 establishes approximations on a time scale O (l/p) and for as 
long as 0 wanders in a domain where A (6) is a stability matrix. Whenever 
0(0) is such that A (0(0)) is a stability matrix, this will be the case on at 
least a time scale of 0(1 / ji) because 9k+\ — 9k is of O(ji). In the special 
circumstance that some stability property can be established, e.g. an average 
based approximation for 0° has some kind of attractor, strictly contained in 
the stability domain , then all approximations established in Theorem 3.4 
hold on the whole time axis. 

3. Summarizing loosely the content of Theorem 3.4 we have that the solutions 
Xk, 9k of the adaptive system (3.1) 

xjt+i = A(9k)xk + B k (9k), *o, p ^ 

9k i t =9k + ngk(0k, Xk), 0o, 

are O(ji) approximated on a time interval 0(1 /pi) by x g (9 g ), 0® where x° 
is defined via the difference equation (3.3) (the so-called frozen system) 
and 0° is defined in (3.7). 

Xk+iiv) = A(v)x£(v) + B k (v), Xq(v)= xo, 

e k+\ = e k+ • *°(0jt))> 



C.4 Averaging Approximation 



Theorem 3.4 establishes a finite time decoupling of the x and 0 equations in the 
adaptive system (2.3), whereby the 0 variable is approximated by 0° governed 
by the difference equation (3.7). This is in standard form for the application of 
the averaging results discussed in Section C.2. Using the results of Theorems 
2.2 and 2.4 we can obtain the following characterization of the solutions of the 
adaptive system (2.3). 

Theorem 4.1. Consider the adaptive system (2.3), the frozen system (3.3) and 
the approximate update (3.7) under the Assumptions 3.1 and 3.2. Let 0o e . Let 
$8 ■= inf VS 3( ) ||0o - v||. 

Let S = max (||x 0 || , sup ve sup*. ||^(v)||). 

Assume that the function gk (v, x*? (v)) has a well defined average g a (v) for any 
v e with associated order function 8 g (p): 



SgQi) = 



sup sup ll\\Y\gi(v, xf (v)) - g" (v) I 
ve (r) k€[0,L/ui II ;=0 L J 



(4.1) 



There exists a positive constant p. a (Se, S) such that for all ji e [0, pi?) the solu- 
tion Xk, 9k of the adaptive system (2.3) is approximated on a time scale of 0(1 / p) 
by x]?(0* ), 0* up to order 0(S g (p)) where 9 a is defined via: 



0 k+\ = e k + dg a (9f), 9 a 



^0 - 



k = 0,1. 



(4.2) 
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The above result can be extended to an infinite time result provided the aver- 
aged equation has some extra stability property. 

Theorem 4.2. Consider the adaptive system (2.3), the frozen system (3.3) and 
the approximate update (3.7) under Assumptions 3.1 and 3.2. Let 9o e . Let 
8g := inf ve3 ( ) ||0o - v||. 

Let S = max (||x 0 || , sup v€ sup* ||fl*(v)||). 

Assume that the function gk{v, x°(k, v)) has a well defined uniform average 
g a (v) for any v e with associated order function <5“ (/z): 

I ko+k it 

V] [gf(v,xf(v)) -g a (v)l . (4.3) 

i=k 0 11 

Let 0 O o e be a uniformly asymptotically stable equilibrium for the averaged 

equation (4.2) such that Assumption 2.3 holds. Denote by ©oo the largest domain 
of attraction of 9^ fully contained in *. Let 9 q £ ©oo- 

There exists a positive constant /x a (©oo- 3) such that for all /x £ [0, /x a ) the 
solution Xk, 9k of the adaptive system (2.3) is approximated uniformly in k by 
x^(9f), 9£ up to order o(l) where 9 a is defined by equation (4.2). 

Moreover, if the equilibrium 9^ is locally exponentially stable (i.e. all the 
eigenvalues of Dg a (0 oo ) have a negative real part) then the approximation er- 
ror is 0(8g(p)). 

Remark. The statements of Theorems 4.1 and 4.2 lead to the following impor- 
tant conclusion. If the averaged equation (4.2) which captures the essence of the 
adaptation mechanism has no attractor in the domain where the frozen system is 
well defined, that is the adaptive mechanism forces the adapted variable 9 outside 
, then unacceptable behavior is to be expected. In this situation averaging can 
only be applied on a finite time basis and predicts that the adaptive system will 
have poor performance. Indeed as 9 leaves the stability domain the x variable 
will grow exponentially. Whenever there is an attractor in the domain where 
the frozen system behaves well, averaging can be used for the whole trajectory, 
hence may be used to analyze the asymptotic performance of the overall adaptive 
system (2.3). In this case good performance may be achieved. 

It transpires that adaptive algorithm design may concentrate on providing the 
average g a , see (4.2), with the right properties, namely an attractor close to the 
points for which the frozen system has the behavior we would like to see. 



^This is the set of all initial conditions for which the trajectories start in , remain in and 
converge to (?oo- 
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serial port operating system, see SPOS 

series connection, 25 

servo system, 273 

settling time, 93 

sign matrix, 298 

signal disturbance, 2, 59 

signature, 300 

similar matrices, 300 

similarity transformation, 300 

simulation languages, 16 

simultaneous stabilization, 53 

single stepping, 256 

singular 

matrix, 298 
value, 301, 308 
decomposition, 301 
maximum, 310 
sinusoidal disturbance, 60 
skew 

Hermitian, 297 
symmetric, 297 
small gain stability, 30 
small order, 315 
software 

environment 
debugging, 255 
development, 252 
platform, 264 
space 

Hardy 2-, 309 
Hilbert, 309 
image, 299 
Lebesque 2-, 309 
range, 299 
row, 303 
vector, 303 
dual, 304 
specification 

in oo-norm, 98 
using 2-norms, 95 
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spectral factorization, 37 
spectrum, frequency, 62 
discrete, 61 
SPOS, 255 

spread of the interpolating function, 
237 

stability, 311 

asymptotic, 23, 30, 309, 311 
exponential, 311 
global, 311 
BIBO, 309 

of nonlinear systems, 310 
properties, 219 
robust, 19 
small gain, 30 

stabilizability, 37, 102, 211, 306 
stabilizable, 22 
stabilization 
result, 224 
robust, 75, 90 
simultaneous, 53 
stabilizing 

controller, 14, 28 
feedback, 30 

feedforward/feedback, 32 
optimizing, 15 
regulator, 15 
regulators, 51, 53 
state feedback, 37 
stable, 305 

internally, 31, 150, 224 
invariant subspace, 85, 87 
linear fractional representation, 
82 
modes 

uncontrollable, 26 
unobservable, 26 
strongly, 153 
standard form, 316 
state, 4, 10 

equation, 305 
estimate feedback, 104 
controller, 37 
space 

equation, 19 



representation, 81 
steepest descent, 161, 164, 169 
step response specification, 93 
stochastic, 4 

disturbance, 239 
model, 61 
excitation, 12 
system, 7 

strictly proper, 308 

transfer function, 22 
strongly stable, 153 
structural resonance mode, 289 
structured uncertainties, 15 
subparitioning, 23 
subset, interior, 320 
subspace, 303 

stable invariant, 85, 87 
vector, 303 
sufficiently 

exciting, 6 
rich signals, 162 
sum, direct, 297 
surjective, 304 

Sylvester’s Inertia Theorem, 300 
symmetric, 297 
skew, 297 
system 

dynamical, 2, 305 
linear, 7, 305 
finite-dimensional, 7 
frozen, 321, 322 
stochastic, 7 
time-invariant, 7 
time- varying, 53, 212 

Taylor series expansion, 208 
three-term-controller, 3 
time 

-invariant system, 7 
-varying system, 53, 212 
domain uncertainty, 68 
scale 

approach, 12 
separation, 313 
trace, 299 
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tracking, 10 
LQ, 95 

trajectory selection, 237 
transfer function, 19, 305 
proper, 22 
rational, 22 
strictly, 22 
transformation 

coordinate basis, 23, 26 
linear, 304 
similarity, 300 
transient response, 93 
transpose, 25, 297 
truncated Gaussian, 23 1 
truncation, balanced, 308 
tunability, 188 
tuning property, 188, 189 
with excitation, 189 
two-degree-of-freedom controller, 20, 
33,53 
2-norm, 301 

specification using, 95 
worst case design using, 97 
2-space 

Hardy, 309 
Lebesque, 309 

UIO, 265 

undershoot, maximum, 93 
uniform 

average, 319 

complete controllability, 307 
unit step, 93 
unitary matrix, 298 
universal input-output, see UIO 
unmodeled 

disturbance, 210 
dynamics, 2, 8, 167, 210, 239 
unstructured uncertainties, 15 
up/down counter, 247 
Usenet, 265 

variation of constants formula, 160 
vector, 297 
basis, 303 



norm, 301 
space, 303 
subspace, 303 

voltage to current converter, 247 

well-posed, 150, 224 
white noise, 62, 101 
disturbance, 10 
wing flutter, 289 
worst case 

design using 2-norms, 97 
disturbance, 10 

Z-transform, 305 
zero 

complex, 299 
real, 299 




